Cipher Cracking 2013 - Revision history

A1195007: /* See also */

2013-10-24T02:22:47Z

A1195007: /* See also */

2013-05-15T02:16:39Z

Dabbott: /* See also */

2013-03-19T03:28:53Z

Dabbott: /* Semester B */

2013-03-19T02:13:26Z

Semester B

← Older revision		Revision as of 11:43, 19 March 2013
Line 58:		Line 58:
	* Project exhibition 'expo' (Friday, Week 12)		* Project exhibition 'expo' (Friday, Week 12)
	* CD or stick containing your whole project directories (Tuesday, Week 13)		* CD or stick containing your whole project directories (Tuesday, Week 13)
	* YouTube video (Tuesday, Week 13)		* YouTube video (Tuesday, Week 13) - add the URL to this wiki

	== Weekly progress and questions ==		== Weekly progress and questions ==

Dabbott: /* Specific tasks */

2013-03-19T01:28:22Z

Specific tasks

← Older revision		Revision as of 10:58, 19 March 2013
Line 43:		Line 43:

	* Plot, present, and interpret the mass spectrometer data we have.		* Plot, present, and interpret the mass spectrometer data we have.

	* Create an entertaining YouTube video of your whole project.

	==Deliverables==		==Deliverables==

Dabbott: /* Specific tasks */

2013-03-19T01:26:12Z

Specific tasks

← Older revision		Revision as of 10:56, 19 March 2013
Line 28:		Line 28:
	Here are the remaining tasks resulting from previous work. You may want to focus on a subset of these:		Here are the remaining tasks resulting from previous work. You may want to focus on a subset of these:

	# Critically review the statistical analysis of the letters. See if you can extend it (eg. testing another language previous students missed and checking if they included all possibilities of ambiguous letters). Is the conclusion of previous team correct? Because there are lots of interesting new tasks (see below) don't spend too long on this. Just spend enough time on this as a quick warm up exercise.		* Critically review the statistical analysis of the letters. See if you can extend it (eg. testing another language previous students missed and checking if they included all possibilities of ambiguous letters). Is the conclusion of previous team correct? Because there are lots of interesting new tasks (see below) don't spend too long on this. Just spend enough time on this as a quick warm up exercise.

	# Download the searchable pdf file of the copy of the [http://www.eleceng.adelaide.edu.au/personal/dabbott/tamanshud/W&T_rubaiyat_wells_copy.pdf Omar Khayyam] that closely matches the dead man's copy. Create an ascii file with the raw text. Use this as a one-time pad that directly substitutes the letters of the alphabet a-z. The book contains 75 quatrains (four sentence poems) each containing about 140 letters. So the whole book contains about <math>75 \times 140 = 10,500\,</math> letters. As we don't know where in the book the one-time pad starts, start at the beginning and step through the whole book one letter at a time. You'll then end up with >10,000 decrypts. Write a software script to look for the most common top-20 words in all the decrypts to narrow down to a few possible results that can be examined by eye.		* Download the searchable pdf file of the copy of the [http://www.eleceng.adelaide.edu.au/personal/dabbott/tamanshud/W&T_rubaiyat_wells_copy.pdf Omar Khayyam] that closely matches the dead man's copy. Create an ascii file with the raw text. Use this as a one-time pad that directly substitutes the letters of the alphabet a-z. The book contains 75 quatrains (four sentence poems) each containing about 140 letters. So the whole book contains about <math>75 \times 140 = 10,500\,</math> letters. As we don't know where in the book the one-time pad starts, start at the beginning and step through the whole book one letter at a time. You'll then end up with >10,000 decrypts. Write a software script to look for the most common top-20 words in all the decrypts to narrow down to a few possible results that can be examined by eye.

	# Extend the [[Media:Cipher GUI.rar\|CipherGUI 2011]] software that was created by a previous team. See if you can add more ciphers to the collection. Use it to eliminate more ciphers and enter your conclusions here: [[Cipher Cross-off List]]. Be critical and be prepared to question and recheck some of the items already on the list.		* Extend the [[Media:Cipher GUI.rar\|CipherGUI 2011]] software that was created by a previous team. See if you can add more ciphers to the collection. Use it to eliminate more ciphers and enter your conclusions here: [[Cipher Cross-off List]]. Be critical and be prepared to question and recheck some of the items already on the list.

	* A previous team created a webcrawler and search engine to search keywords with wild cards, as Google does not allow this. This is to check for common repeated expressions on the WWW that may contain initial letters that are also in the code. If the code is an initialism, this will give us a clue as to some likely content. There are two things that need to be fixed: (i) we need you to write a convenient web-interface for this search engine, and (ii) we need to scrap the webcrawler as it takes too long. It is impossible to crawl the whole web with one little PC and therefore you need to interface the search engine to operate on an index that has already been created by a commercial webtrawler. Unfortunately, companies like Google won't let you access their index list. Therefore you need to use another provider such as [http://yacy.de/en/index.html YaCy]. You may also want to check out a resource called [http://commoncrawl.org/get-started/ CommonCrawl]. To use CommonCrawl you'll need to sign up with both [https://commoncrawl.atlassian.net/wiki/display/CRWL/Quick+Start+-+Build+from+Github github] (to download the Java source code) and also [http://docs.aws.amazon.com/ElasticMapReduce/latest/GettingStartedGuide/Welcome.html?r=4524 Amazon] (to run your uploaded compiled code).		* A previous team created a webcrawler and search engine to search keywords with wild cards, as Google does not allow this. This is to check for common repeated expressions on the WWW that may contain initial letters that are also in the code. If the code is an initialism, this will give us a clue as to some likely content. There are two things that need to be fixed: (i) we need you to write a convenient web-interface for this search engine, and (ii) we need to scrap the webcrawler as it takes too long. It is impossible to crawl the whole web with one little PC and therefore you need to interface the search engine to operate on an index that has already been created by a commercial webtrawler. Unfortunately, companies like Google won't let you access their index list. Therefore you need to use another provider such as [http://yacy.de/en/index.html YaCy]. You may also want to check out a resource called [http://commoncrawl.org/get-started/ CommonCrawl]. To use CommonCrawl you'll need to sign up with both [https://commoncrawl.atlassian.net/wiki/display/CRWL/Quick+Start+-+Build+from+Github github] (to download the Java source code) and also [http://docs.aws.amazon.com/ElasticMapReduce/latest/GettingStartedGuide/Welcome.html?r=4524 Amazon] (to run your uploaded compiled code).

Dabbott: /* Specific tasks */

2013-03-19T01:25:37Z

Specific tasks

← Older revision		Revision as of 10:55, 19 March 2013
Line 28:		Line 28:
	Here are the remaining tasks resulting from previous work. You may want to focus on a subset of these:		Here are the remaining tasks resulting from previous work. You may want to focus on a subset of these:

	#* Critically review the statistical analysis of the letters. See if you can extend it (eg. testing another language previous students missed and checking if they included all possibilities of ambiguous letters). Is the conclusion of previous team correct? Because there are lots of interesting new tasks (see below) don't spend too long on this. Just spend enough time on this as a quick warm up exercise.		# Critically review the statistical analysis of the letters. See if you can extend it (eg. testing another language previous students missed and checking if they included all possibilities of ambiguous letters). Is the conclusion of previous team correct? Because there are lots of interesting new tasks (see below) don't spend too long on this. Just spend enough time on this as a quick warm up exercise.

	#* Download the searchable pdf file of the copy of the [http://www.eleceng.adelaide.edu.au/personal/dabbott/tamanshud/W&T_rubaiyat_wells_copy.pdf Omar Khayyam] that closely matches the dead man's copy. Create an ascii file with the raw text. Use this as a one-time pad that directly substitutes the letters of the alphabet a-z. The book contains 75 quatrains (four sentence poems) each containing about 140 letters. So the whole book contains about <math>75 \times 140 = 10,500\,</math> letters. As we don't know where in the book the one-time pad starts, start at the beginning and step through the whole book one letter at a time. You'll then end up with >10,000 decrypts. Write a software script to look for the most common top-20 words in all the decrypts to narrow down to a few possible results that can be examined by eye.		# Download the searchable pdf file of the copy of the [http://www.eleceng.adelaide.edu.au/personal/dabbott/tamanshud/W&T_rubaiyat_wells_copy.pdf Omar Khayyam] that closely matches the dead man's copy. Create an ascii file with the raw text. Use this as a one-time pad that directly substitutes the letters of the alphabet a-z. The book contains 75 quatrains (four sentence poems) each containing about 140 letters. So the whole book contains about <math>75 \times 140 = 10,500\,</math> letters. As we don't know where in the book the one-time pad starts, start at the beginning and step through the whole book one letter at a time. You'll then end up with >10,000 decrypts. Write a software script to look for the most common top-20 words in all the decrypts to narrow down to a few possible results that can be examined by eye.

	* Extend the [[Media:Cipher GUI.rar\|CipherGUI 2011]] software that was created by a previous team. See if you can add more ciphers to the collection. Use it to eliminate more ciphers and enter your conclusions here: [[Cipher Cross-off List]]. Be critical and be prepared to question and recheck some of the items already on the list.		# Extend the [[Media:Cipher GUI.rar\|CipherGUI 2011]] software that was created by a previous team. See if you can add more ciphers to the collection. Use it to eliminate more ciphers and enter your conclusions here: [[Cipher Cross-off List]]. Be critical and be prepared to question and recheck some of the items already on the list.

	* A previous team created a webcrawler and search engine to search keywords with wild cards, as Google does not allow this. This is to check for common repeated expressions on the WWW that may contain initial letters that are also in the code. If the code is an initialism, this will give us a clue as to some likely content. There are two things that need to be fixed: (i) we need you to write a convenient web-interface for this search engine, and (ii) we need to scrap the webcrawler as it takes too long. It is impossible to crawl the whole web with one little PC and therefore you need to interface the search engine to operate on an index that has already been created by a commercial webtrawler. Unfortunately, companies like Google won't let you access their index list. Therefore you need to use another provider such as [http://yacy.de/en/index.html YaCy]. You may also want to check out a resource called [http://commoncrawl.org/get-started/ CommonCrawl]. To use CommonCrawl you'll need to sign up with both [https://commoncrawl.atlassian.net/wiki/display/CRWL/Quick+Start+-+Build+from+Github github] (to download the Java source code) and also [http://docs.aws.amazon.com/ElasticMapReduce/latest/GettingStartedGuide/Welcome.html?r=4524 Amazon] (to run your uploaded compiled code).		* A previous team created a webcrawler and search engine to search keywords with wild cards, as Google does not allow this. This is to check for common repeated expressions on the WWW that may contain initial letters that are also in the code. If the code is an initialism, this will give us a clue as to some likely content. There are two things that need to be fixed: (i) we need you to write a convenient web-interface for this search engine, and (ii) we need to scrap the webcrawler as it takes too long. It is impossible to crawl the whole web with one little PC and therefore you need to interface the search engine to operate on an index that has already been created by a commercial webtrawler. Unfortunately, companies like Google won't let you access their index list. Therefore you need to use another provider such as [http://yacy.de/en/index.html YaCy]. You may also want to check out a resource called [http://commoncrawl.org/get-started/ CommonCrawl]. To use CommonCrawl you'll need to sign up with both [https://commoncrawl.atlassian.net/wiki/display/CRWL/Quick+Start+-+Build+from+Github github] (to download the Java source code) and also [http://docs.aws.amazon.com/ElasticMapReduce/latest/GettingStartedGuide/Welcome.html?r=4524 Amazon] (to run your uploaded compiled code).

Dabbott: /* Specific tasks */

2013-03-19T01:25:07Z

Specific tasks

← Older revision		Revision as of 10:55, 19 March 2013
Line 28:		Line 28:
	Here are the remaining tasks resulting from previous work. You may want to focus on a subset of these:		Here are the remaining tasks resulting from previous work. You may want to focus on a subset of these:

	* Critically review the statistical analysis of the letters. See if you can extend it (eg. testing another language previous students missed and checking if they included all possibilities of ambiguous letters). Is the conclusion of previous team correct? Because there are lots of interesting new tasks (see below) don't spend too long on this. Just spend enough time on this as a quick warm up exercise.		#* Critically review the statistical analysis of the letters. See if you can extend it (eg. testing another language previous students missed and checking if they included all possibilities of ambiguous letters). Is the conclusion of previous team correct? Because there are lots of interesting new tasks (see below) don't spend too long on this. Just spend enough time on this as a quick warm up exercise.

	* Download the searchable pdf file of the copy of the [http://www.eleceng.adelaide.edu.au/personal/dabbott/tamanshud/W&T_rubaiyat_wells_copy.pdf Omar Khayyam] that closely matches the dead man's copy. Create an ascii file with the raw text. Use this as a one-time pad that directly substitutes the letters of the alphabet a-z. The book contains 75 quatrains (four sentence poems) each containing about 140 letters. So the whole book contains about <math>75 \times 140 = 10,500\,</math> letters. As we don't know where in the book the one-time pad starts, start at the beginning and step through the whole book one letter at a time. You'll then end up with >10,000 decrypts. Write a software script to look for the most common top-20 words in all the decrypts to narrow down to a few possible results that can be examined by eye.		#* Download the searchable pdf file of the copy of the [http://www.eleceng.adelaide.edu.au/personal/dabbott/tamanshud/W&T_rubaiyat_wells_copy.pdf Omar Khayyam] that closely matches the dead man's copy. Create an ascii file with the raw text. Use this as a one-time pad that directly substitutes the letters of the alphabet a-z. The book contains 75 quatrains (four sentence poems) each containing about 140 letters. So the whole book contains about <math>75 \times 140 = 10,500\,</math> letters. As we don't know where in the book the one-time pad starts, start at the beginning and step through the whole book one letter at a time. You'll then end up with >10,000 decrypts. Write a software script to look for the most common top-20 words in all the decrypts to narrow down to a few possible results that can be examined by eye.

	* Extend the [[Media:Cipher GUI.rar\|CipherGUI 2011]] software that was created by a previous team. See if you can add more ciphers to the collection. Use it to eliminate more ciphers and enter your conclusions here: [[Cipher Cross-off List]]. Be critical and be prepared to question and recheck some of the items already on the list.		* Extend the [[Media:Cipher GUI.rar\|CipherGUI 2011]] software that was created by a previous team. See if you can add more ciphers to the collection. Use it to eliminate more ciphers and enter your conclusions here: [[Cipher Cross-off List]]. Be critical and be prepared to question and recheck some of the items already on the list.

Dabbott: /* Specific tasks */

2013-03-19T01:23:19Z

Specific tasks

← Older revision		Revision as of 10:53, 19 March 2013
Line 34:		Line 34:
	* Extend the [[Media:Cipher GUI.rar\|CipherGUI 2011]] software that was created by a previous team. See if you can add more ciphers to the collection. Use it to eliminate more ciphers and enter your conclusions here: [[Cipher Cross-off List]]. Be critical and be prepared to question and recheck some of the items already on the list.		* Extend the [[Media:Cipher GUI.rar\|CipherGUI 2011]] software that was created by a previous team. See if you can add more ciphers to the collection. Use it to eliminate more ciphers and enter your conclusions here: [[Cipher Cross-off List]]. Be critical and be prepared to question and recheck some of the items already on the list.

	* A previous team created a webcrawler and search engine to search keywords with wild cards, as Google does not allow this. This is to check for common repeated expressions on the WWW that may contain initial letters that are also in the code. If the code is an initialism, this will give us a clue as to some likely content. There are two things that need to be fixed: (i) we need you to write a convenient web-interface for this search engine, and (ii) we need to scrap the webcrawler as it takes too long. It is impossible to crawl the whole web with one little PC and therefore you need to interface the search engine to operate on an index that has already been created by a commercial webtrawler. Unfortunately, companies like Google won't let you access their index list. Therefore you need to use another provider such as [http://yacy.de/en/index.html YaCy]. You may also want to check out a resource called [http://commoncrawl.org/get-started/ CommonCrawl].		* A previous team created a webcrawler and search engine to search keywords with wild cards, as Google does not allow this. This is to check for common repeated expressions on the WWW that may contain initial letters that are also in the code. If the code is an initialism, this will give us a clue as to some likely content. There are two things that need to be fixed: (i) we need you to write a convenient web-interface for this search engine, and (ii) we need to scrap the webcrawler as it takes too long. It is impossible to crawl the whole web with one little PC and therefore you need to interface the search engine to operate on an index that has already been created by a commercial webtrawler. Unfortunately, companies like Google won't let you access their index list. Therefore you need to use another provider such as [http://yacy.de/en/index.html YaCy]. You may also want to check out a resource called [http://commoncrawl.org/get-started/ CommonCrawl]. To use CommonCrawl you'll need to sign up with both [https://commoncrawl.atlassian.net/wiki/display/CRWL/Quick+Start+-+Build+from+Github github] (to download the Java source code) and also [http://docs.aws.amazon.com/ElasticMapReduce/latest/GettingStartedGuide/Welcome.html?r=4524 Amazon] (to run your uploaded compiled code).

	* Use computer graphics to reconstruct and undistort the face of the dead man. What would he look like if he were alive? To do this you need the data from the previous group that scanned the man's face from a plaster bust, at the Police Museum, with a [http://www.david-laserscanner.com/ 3D scanner]. An example of the type of graphics software you can use to manipulate the scanned image is [http://www.123dapp.com/catch 123D]. You may want to investigate other 3D rendering graphics software.		* Use computer graphics to reconstruct and undistort the face of the dead man. What would he look like if he were alive? To do this you need the data from the previous group that scanned the man's face from a plaster bust, at the Police Museum, with a [http://www.david-laserscanner.com/ 3D scanner]. An example of the type of graphics software you can use to manipulate the scanned image is [http://www.123dapp.com/catch 123D]. You may want to investigate other 3D rendering graphics software.

Dabbott: /* References and useful resources */

2013-03-19T01:16:41Z

References and useful resources

← Older revision		Revision as of 10:46, 19 March 2013
Line 88:		Line 88:
	If you find any useful external links, list them here:		If you find any useful external links, list them here:
	* [http://en.wikipedia.org/wiki/Taman_Shud_Case The taman shud case]		* [http://en.wikipedia.org/wiki/Taman_Shud_Case The taman shud case]
			* [http://yacy.de/en/index.html YaCy]
			* [http://commoncrawl.org/get-started/ CommonCrawl].
	* [http://www.library.cornell.edu/colldev/mideast/okhym.htm Edward Fitzgerald's translation of رباعیات عمر خیام by عمر خیام]		* [http://www.library.cornell.edu/colldev/mideast/okhym.htm Edward Fitzgerald's translation of رباعیات عمر خیام by عمر خیام]
	* [http://ebooks.adelaide.edu.au/ Adelaide Uni Library e-book collection]		* [http://ebooks.adelaide.edu.au/ Adelaide Uni Library e-book collection]

← Older revision		Revision as of 11:52, 24 October 2013
Line 83:		Line 83:
	* [[Semester B Final Report 2013 - Cipher cracking]]		* [[Semester B Final Report 2013 - Cipher cracking]]
	* [[Cipher Cross-off List]]		* [[Cipher Cross-off List]]
			* [[File:Cipher_GUI.rar]]
			* [[File:DecodingToolkit.rar]]

	== References and useful resources==		== References and useful resources==

← Older revision		Revision as of 11:46, 15 May 2013
Line 77:		Line 77:

	==See also==		==See also==
			* [[Cipher_Cracking_2013/Rubaiyat_Plaintext]]
	* [[Cipher Cracking]] (main page)		* [[Cipher Cracking]] (main page)
	* [[Cipher cracking 2013 weekly progress]]		* [[Cipher cracking 2013 weekly progress]]

← Older revision		Revision as of 12:58, 19 March 2013
Line 79:		Line 79:
	* [[Cipher Cracking]] (main page)		* [[Cipher Cracking]] (main page)
	* [[Cipher cracking 2013 weekly progress]]		* [[Cipher cracking 2013 weekly progress]]
	* [[Semester A Progress Report 2013]]		* [[Semester A Progress Report 2013 - Cipher cracking]]
	* [[Semester B Final Report 2013]]		* [[Semester B Final Report 2013 - Cipher cracking]]
	* [[Cipher Cross-off List]]		* [[Cipher Cross-off List]]