Python Web Request (web page batch download system)
"For downloading large amounts of swf archives"
Flash will be closed in 2021, but the offline version of flash can still drive relationships, so I want to download a large number of flash files at a time and convert them into html5/mp4/gif files, etc. to give them the opportunity to be reborn.
This system is aimed at .swf works at https://dagobah.net/. The main reasons are as follows:
- The archive download of dagobah.net is regular. You can download the archive by changing "flash" to "flashswf"
- There is no rate-limited limit, so you can download unlimitedly large amounts of money without being cut off by the website.
- Most flashes have not been watched before, so you need to download them to taste them slowly
- Transferring to other formats gives these works a chance to be reborn
Information Archives
This time, the python archive is divided into 4, and if you have the opportunity, you can make it into one archive:
- Used to intercept the html screen of 275 pages of dagobah.net and pull them all down
- Filter out most of the unimportant html and only keep the line that has *.swf
- The barcode that pure*.swf is pulled out from that line
- Stuff in the pre-prepared download prefix URL and then download for loop
Pre-installation
No, just have python and hands
Effect preview
Total time spent: 16小时
It is related to a large number of archives, so the time will really be delayed for a long time.
You will get about 13,876 swf files, 30% of which are all 3D dragon effects. Please pay attention to patients with epilepsy.
Extended ideas
- Is there a way to merge these four files into one?
- If the web page to be archaeological this time is https://z0r.de, how do you modify the code?
References
?