Editing Final Report 2011 (section)

===Web Crawler===

There are several directions that future projects can extend the combined pattern matching web crawling software developed in 2011. A natural progression envisaged by the current project team is to further enhance the data mining capabilities of the web crawler. This could be achieved by using the distributed research computing system of the University of Adelaide. A reconfiguration of the current web crawler, which operates using a single computer, would be required. Another advantage to this proposal is that the storage capacity for search results would be greatly improved enabling more data to be collected. With an eye to the future, the rapidly increasing levels of images used on the web suggest implementing image analysis into the crawler could also be an innovative idea.

Another possible extension to the web crawler component of the project is the implementation of processing algorithms to analyse collected search data. For patterns that occur more commonly within English text this will provide an efficient method of extracting useful information from the acquired data. The code investigation process could be further streamlined by including statistical analysis capabilities into these analytical algorithms.

In regards to the use of the web crawler to investigate the Somerton Man code, much potential work remains. This year, only a brief prototype-style investigation was launched due to time constraints towards the end of the project. There remains the possibility that extensive crawling for patterns could unearth the hidden message behind the Somerton Code. As such, a more intensive data gathering investigation could be launched including a statistical analysis of results using the Web Crawler produced this year.