Skip to content

A new web archive tool!

I have been more and more interested in the history of the web, and development of websites. But when i look around, there aren’t many tools that is dedicated to archive the web, so when a website goes down, it is deleted for ever, and may not ever be seen again. For example, do you remember how microsoft.com, apple.com or google.com looked like in 2001? Me neither, and the only good tool out there is The Wayback Machine. There are other sites that archive the web, such as UK Web Archive and Archive-It, but i feel that i want to make my own tool.

The site i have made can be found at https://archive.gorgonsteel.io/ and are still under development. All the other archives keep a copy of the site itself, while mine is based on taking a complete screenshot of the whole page. It will take months to even be worth mentioning as a website archive, but i hope i will be able to make this tool useful enough to help out a few people. As for now, it just cache one screenshot of each website, but in the near future will i build out a archive tool, where you can search and add websites, as well as looking back at old screenshots.

As for now, all screenshots are stored on a 2 TB hard drive, but i feel that i will run out of disk space within a few months if this grow. I will also build a tool to take one screenshot of every website submitted to the archive each day, and when the site get larger, i will shrink it down to once every 3 days. Right now, the webserver has 8 GB of RAM, witch should be enough to run this site smooth and if it need more ram or storage, i will provide this at my own cost to make this project as successful as possible. I will also try to set up some sort of backup, and i may set up a way for people to remote connect to my server through FTP and take backups of the screenshots to make sure that they wont get lost in case of a crash or failed hard drive.

Hope this will work out, wish me good luck!

Leave a Reply

Your email address will not be published. Required fields are marked *