43 lines
1.4 KiB
Markdown
43 lines
1.4 KiB
Markdown
# Fun with wget
|
||
|
||
## wget multiple files
|
||
|
||
```bash
|
||
wget -r -l1 -A.mp3 http://aaa.com/directory
|
||
```
|
||
|
||
In the above example, `-r` and `-l1` options together enable 1-level deep recursive retrieval, and `-A` option specifies lists of file name suffixes to accept during recursive download (`.mp3` in this case).
|
||
|
||
## ways to wget entire webpage
|
||
This one works well, I've created an alias for it:
|
||
```bash
|
||
wgetMirror='/usr/bin/wget -o wget.log -mkEpnp –wait=9 –user-agent='\''Mozilla/5.0 (compatible; Googlebot/2.1; +http://www. google.com/bot.html)'\'' –no-check-certificate'
|
||
```
|
||
|
||
Other options:
|
||
```bash
|
||
wget --recursive --no-clobber --page-requisites --html-extension --convert-links --restrict-file-names=windows --domains example.com --no-parent <url>
|
||
```
|
||
|
||
|
||
## Modify User Agent
|
||
|
||
```bash
|
||
$ wget -U "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" <url>
|
||
```
|
||
|
||
## wget from webdav with authentication
|
||
|
||
```bash
|
||
wget --http-user=user-id --http-password=password <URL>
|
||
```
|
||
|
||
|
||
---
|
||
|
||
## Reference
|
||
|
||
* [Download Multiple Files with wget](http://xmodulo.com/how-to-download-multiple-files-with-wget.html)
|
||
* [Archiving website with wget](https://www.petekeen.net/archiving-websites-with-wget)
|
||
* [Download entire webpage with wget](https://janezurevc.name/download-entire-web-page-using-wget)
|
||
* [Change the User Agent in wget](https://www.networkinghowtos.com/howto/change-the-user-agent-in-wget/) |