Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision | |||
scripts:working_with_websites [2022-04-12 19:11] – Tony | scripts:working_with_websites [2022-06-12 04:14] (current) – Tony | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ====== Working with Websites ====== | ||
+ | |||
+ | |||
+ | Some great tools for web or internet stuff I like. | ||
+ | |||
+ | |||
+ | ====== Tools ====== | ||
+ | |||
+ | |||
+ | ===== Downloading ===== | ||
+ | |||
+ | * curl (great for testing HTTP requests) | ||
+ | * aria2c (great for faster downloads via multiple threads) | ||
+ | * httrack (mirrors websites but really slow) | ||
+ | * youtube-dl (download or stream media) | ||
+ | * rtmpdump (or just use youtube-dl) | ||
+ | |||
+ | |||
+ | ===== Monitoring ===== | ||
+ | |||
+ | * vnstat (checking network usage) | ||
+ | * nmap (local subnet IP searching) | ||
+ | * ipscan (simple java IP scanner) | ||
+ | * iperf (custom link speed test) | ||
+ | |||
+ | ===== Handy tools ===== | ||
+ | |||
+ | * iodine (handy VPN over DNS for restricted networks) | ||
+ | * sshuttle (tunnel network traffic over SSH) | ||
+ | * wireguard (do i need to mention this? fast, safe vpn protocol) | ||
+ | * Postman (for testing REST API's without memorizing cURL) | ||
+ | |||
+ | |||
+ | ====== Scripts ====== | ||
+ | |||
+ | |||
+ | ===== Mirror all files from a website ===== | ||
+ | |||
+ | Mirror all the files off of a website, with the full folder structure | ||
+ | |||
+ | wget -m -p -E -k -K -np https:// | ||
+ | |||
+ | |||
+ | ===== Save websites as shortcuts ===== | ||
+ | |||
+ | On Windows, you can save links to websites. In Linux, Ctrl+S and drag/drop just save the HTML. | ||
+ | |||
+ | Well, turns out you can make .desktop files to just link to websites. Here's the jankjank script | ||
+ | to easily make desktop files. | ||
+ | |||
+ | < | ||
+ | echo " | ||
+ | echo " | ||
+ | echo " | ||
+ | echo " | ||
+ | echo URL=" | ||
+ | echo " | ||
+ | </ | ||
+ | |||
+ | Parameter 1 is the name of the shortcut and file, parameter 2 is the actual URL. Change icon and other params as you see fit. | ||
+ | |||
+ | Save as a bash function or as a script. Usage: | ||
+ | bash link.sh " | ||
+ | | ||
+ | | ||
+ | ===== Tunneling stuff ===== | ||
+ | |||
+ | ==== Files over SSH ==== | ||
+ | * SFTP | ||
+ | * SCP | ||
+ | * SSHFS (FUSE for SFTP) | ||
+ | * Rsync | ||
+ | |||
+ | ==== Others ==== | ||
+ | * SSHuttle (VPN over SSH) | ||
+ | * NetCat (just yeet stuff across systems) | ||
+ | * Rclone (for other cloud) | ||
+ | * S3FS (for when I need more disk space through B2) | ||
+ | |||
+ | |||
+ | |||
+ | |||