Web Crawler 2011

Web Crawler

This page contains instructions for downloading and installing the Web Crawler created in the 2011 project, Code Cracking: Who murdered the Somerton Man? (Cipher Cracking 2011).

Instructions

1. Download the Web Crawler Graphical User interface, found here.
2. Extract to your desired location. You should have a folder labelled Crawler, containing a java archive file, Crawler.jar, and a README.txt file (containing instructions for use of the crawler).
3. This Crawler.jar file is an executable file so can be run through double click. However it requires some libraries. These libraries can be found on the crawler4j site which should be the top link from Googling "crawler4j". Download the crawler4j-dependencies-lib.zip libraries as shown in the image below.

crawler4j website with the dependencies library download (click to enlarge).

1. Extract the downloaded libraries into your Crawler folder you have just formed in Step 2. It should appear as a folder titled "lib".
2. Change the name of the folder from "lib" to "Crawler_lib".
3. You are now ready to run the Web Crawler! Simply double click the Crawler.jar file and providing the above steps have been followed correctly, the crawler should function. From this point on, for instructions, see the README.txt file.