Difference between revisions of "Cipher cracking 2018 weekly progress"

From Derek
Jump to: navigation, search
(→‎Semester 1 Week 7)
(→‎Weekly progress)
 
(33 intermediate revisions by the same user not shown)
Line 4: Line 4:
  
 
# Week 1 progress:
 
# Week 1 progress:
−
#* Organise meeting with Professor Abbott.
+
#* Organise meeting with Prof. Abbott.
 +
#* Contacted project partner and had a meeting.
 +
#* Attended the literature training session with project partner at Bar Smith library.
  
 
# Goals for Week 2:
 
# Goals for Week 2:
 +
#* Meet Prof. Abbott.
 +
#* Understand the aims of the project
  
 
=== Semester 1 Week 2===
 
=== Semester 1 Week 2===
  
 
# Week 2 progress:
 
# Week 2 progress:
 +
#* Met with Prof. Abbott.
 +
#* Conducted Background research by reading about previous final year projects.
  
 
# Goals for Week 3:
 
# Goals for Week 3:
−
 
+
#* Start preparing for the Proposal Seminar.
 +
#* Research about Somertons Mans letters.
  
 
=== Semester 1 Week 3===
 
=== Semester 1 Week 3===
Line 19: Line 26:
  
 
# Week 3 progress:
 
# Week 3 progress:
 +
#* Start Work Break Down Structure
 +
#* Commenced work on project Gantt Chart
 +
#* Organised meeting with Professor Abbott
 +
#* Worked on Proposal
 +
#* Met with project partner
  
 
# Goals for Week 4:
 
# Goals for Week 4:
 +
#* Attend meeting with Professor Abbott
 +
#* Finish project Gantt Chart
  
 
=== Semester 1 Week 4===
 
=== Semester 1 Week 4===
Line 26: Line 40:
  
 
# Week 4 progress:
 
# Week 4 progress:
 +
#* Finished project Gantt Chart
 +
#* Met with Professor Abbott
 +
#* Worked on Proposal
 +
#* Met with project partner
 +
#* MATLAB coding started for Task 1
  
 
# Goals for Week 5:
 
# Goals for Week 5:
−
 
+
#* Finish Power point for proposal
 +
#* Make more progress in with Task 1
  
 
=== Semester 1 Week 5===
 
=== Semester 1 Week 5===
  
 
# Week 5 progress:
 
# Week 5 progress:
 +
#* Power Point nearly completed
 +
#* Names of horses, city names etc. have been finalized
 +
#* Task 1 about 50% complete
  
−
# Goals for Week 6:
+
# Goals for Mid-Semester Break:
 +
#* Power Point has to be completed
 +
#* Prepare speech for proposal
 +
#* Meet with project partner in break to complete Task 1
  
 +
=== Semester 1 Mid-Semester Break===
  
−
=== Semester 1 Week 6===
 
  
 +
# Mid-Semester Break progress:
 +
#* Met with project partner
 +
#* More coding done for Task 1
 +
#* Graphs have been produced for Task 1
 +
#* Task 1 75% complete
 +
#* Power Point got drafted by Prof. Abbott
 +
#* Proposal seminar completed
 +
#* DNA testing kit has been delivered
  
−
# Week 6 progress:
+
# Goals for Week 6:
 +
#* Organise meeting with Prof. Abbott
 +
#* Complete Task 1
 +
#* Collect the DNA kit
 +
#* Send DNA kit ASAP
  
−
# Goals for Mid-Semester Break:
+
=== Semester 1 Week 6===
  
  
−
 
+
# Week 6 progress:
−
 
+
#* Sent DNA to 23andMe
−
=== Semester 1 Mid-Semester Break===
+
#* Had meeting with Derek
−
 
+
#* An extension of Task 1 was given, to further look into the book
−
 
+
#* Still working on Task 1
−
# Mid-Semester Break progress:
+
  
 
# Goals for Week 7:
 
# Goals for Week 7:
−
 
+
#* Work on the extension task
 +
#* Start working on thesis
  
 
=== Semester 1 Week 7===
 
=== Semester 1 Week 7===
Line 59: Line 97:
  
 
# Week 7 progress:
 
# Week 7 progress:
−
 
+
#* Original of Task 1 is completed
 +
#* Extension of Task 1 near completion
 +
#* The code is complete to count the letter in each line of the book
 +
#* Thesis layout completed
 +
#* Thesis introduction started
 +
 
# Goals for Week 8:
 
# Goals for Week 8:
 +
#* Complete extension of Task 1
 +
#* Complete Thesis introduction
 +
#* Oragnize meeting with Derek
  
 
=== Semester 1 Week 8===
 
=== Semester 1 Week 8===
  
 
# Week 8 progress:
 
# Week 8 progress:
−
#* Met with Professor Abbott.
+
#* Meeting with Derek
−
#* Met with Jikai to discuss progress.
+
#* Derek said that we have good progress
−
#* Evaluated usability of 2013 group’s statistical analysis for extension: Decided to use a combination of Java and Excel.  Java to be used for text processing, and outputting letter frequency results, and Excel to perform statistical analysis calculations and graph outputs.
+
#* Meeting with partner, to talk about thesis
 +
#* Task 1 is fully completed
 +
#* Results show that horse names are not the mysterious code
  
 
# Goals for Week 9:
 
# Goals for Week 9:
−
#* Organise meeting with Jikai and Professor Abbott.
+
#* Work on thesis
−
#* Contact Peter Varsos
+
−
#* Download programming resources
+
−
#* Begin compiling and formatting text resources for use in statistical frequency analysis.
+
−
 
+
−
 
+
−
# Week 8 progress:
+
−
#* Met with Nick to discuss P-value task progress.
+
−
#* Used excel to calculated P-value for English and Scots.
+
−
 
+
−
 
+
−
# Goals for Week 9:
+
−
#* Making a meeting with Professor Abbott.
+
−
#* Wait for Peter's reply about statistic value for each language in text resources.
+
−
#* Finish the P-value test between English and Somerton code.
+
  
 
=== Semester 1 Week 9===
 
=== Semester 1 Week 9===
Line 90: Line 124:
  
 
# Week 9 progress:
 
# Week 9 progress:
−
#* Organised and attended meeting with Professor Abbott for Friday 15th May
+
#* DNA results have come back, need to have meeting with Derek ASAP
−
#* Researched how to create branches and commit to GitHub
+
#* Thesis introduction completed
−
#* Committed 2013 group's work to GitHub
+
#* Sent drafts to each other
−
#* Downloaded and installed Eclipse
+
−
#* Emailed Peter Varsos about obtaining frequency analysis text files
+
 
   
 
   
 
# Goals for Week 10:
 
# Goals for Week 10:
−
#* Hopefully obtain text files from 2013 group for use in frequency analysis
+
#* Have meeting with Derek
−
#* Prepare test text files to test statistical analysis P-value method
+
#* Continue working on the thesis
−
 
+
−
 
+
−
# Week 9 progress:
+
−
#* Met with Professor Abbott and discuss our progress.
+
−
#* Complete the P-value test between English and Somerton code.
+
−
 
+
−
# Goals for Week 10:
+
−
#* Begin the P-value test for English, French and German.
+
−
#* Compare the 44 letters in somerton code with letters in English, French and German by Declaration of the human rights.
+
  
 
=== Semester 1 Week 10===
 
=== Semester 1 Week 10===
Line 113: Line 136:
  
 
# Week 10 progress:
 
# Week 10 progress:
−
#* Received reply from Peter Varsos, directed me to contact Lucy Griffith
+
#* Meeting with Derek
−
#* Emailed Lucy Griffith and received a reply, awaiting an additional reply to see whether or not she can find original text files for 2013 group’s statistical frequency analysis
+
#* Informed that we can't do any of the DNA task until the ethics is signed
−
#* Received reply from Lucy Griffith with zip file including all text files
+
#* Showed Derek results for task 1 and told us it was good
−
#* Prepared test text files for English, German and French to test our p-value statistical frequency analysis code
+
#* Drafted each others thesis and sent it back to each other
−
#* Committed working version and modifications of 2013’s group to Git Hub for use with text processing
+
−
#* Organised and attended meeting with Professor Abbot for Friday 22nd May
+
−
+
−
# Goals for Week 11:
+
−
#* Research how to create and edit wiki page
+
−
#* Create Research Project Proposal & Progress Report 2015 Wiki page
+
−
#* Download Anova package for Excel to test Jikai’s P-value method
+
−
#* Run test case for P-value in Matlab to check method against Excel and by hand
+
−
#* Commence writing progress report section of Research Project Proposal & Progress Report document
+
−
#* Organise meeting with Professor Abbott for Friday 29th May
+
−
 
+
−
 
+
−
# Week 10 progress:
+
−
#* Received test text files for English, German and French from Nick, and use excel to calculate the P-value for each language declaration against Code, other languages declaration against English statistics and English decalaration against itself.
+
−
#* Meet Nick and Professor Abbot for Friday 22nd May.
+
  
 
# Goals for Week 11:
 
# Goals for Week 11:
−
#* Use mathmatic methed to manually calculate P-value to check the result from excel is correct or not.
+
#* Only task to do now is the thesis
−
#* Use Matlab methed to check the result from excel is correct or not.
+
  
 
=== Semester 1 Week 11===
 
=== Semester 1 Week 11===
Line 141: Line 148:
  
 
# Week 11 progress:
 
# Week 11 progress:
−
#* Researched how to create and edit wiki page
+
#* Thesis introduction completed again
−
#* Created Research Project Proposal & Progress Report 2015 Wiki page
+
#* Thesis technical background started
−
#* Download Stat Plus for Mac to run Anova P-value test in Excel to test Jikai’s P-value method
+
#* Thesis project management started
−
#* Ran test case for P-value in Matlab to check method against Excel and by hand
+
−
#* Commenced writing progress report section of Research Project Proposal & Progress Report document
+
−
#* Attended meeting with Professor Abbott on Friday 29th May
+
 
   
 
   
 
# Goals for Week 12:
 
# Goals for Week 12:
−
#* Prepare text files for statistical analysis
+
#* Have the thesis completed by Tuesday, so we can send our drafts to each other
−
#* Complete Task 1: Statistical Frequency Analysis of Letters
+
−
#* Complete and submit Research Project Proposal & Progress Report onto MyUni by Friday 5th June
+
−
 
+
−
 
+
−
# Week 11 progress:
+
−
#* Used mathmatic methed to manually calculated P-value, the result was same as excel did.
+
−
#* Use Matlab to calculated the P-value, the result was same as excel did.
+
−
#* Attended meeting with Professor Abbott and Nick on Friday 29th May.
+
−
# Goals for Week 12:
+
−
#* Calculate the P-value for European languages form Declaration against Somerton code and compare results with 2013 group's.
+
−
#* Finish Semester A proposal and progress report.
+
  
 
=== Semester 1 Week 12===
 
=== Semester 1 Week 12===
Line 166: Line 159:
  
 
# Week 12 progress:
 
# Week 12 progress:
−
#* Emailed Lucy Griffith for past code or excel spreadsheets used to perform the 2013 group's statistical analysis and received excel spreadsheets containing raw data and method
+
#* Thesis completed and sent to partner
−
#* Emailed Matthew Berryman for assistance with statistical analysis
+
#* Draft each others thesis
−
#* Prepared text files for statistical analysis
+
#* Final copy of draft thesis completed and sent to my uni
−
#* Carried out statistical analysis using Normal Distribution methods
+
−
#* Carried out statistical analysis using Non-Parametric methods
+
−
#* Modified statistical analysis method to obtain more accurate results
+
−
#* Completed Task 1: Statistical Frequency Analysis of Letters
+
−
#* Completed and submitted Research Project Proposal & Progress Report onto MyUni on Friday 5th June
+
  
 
# Goals for Week 13:
 
# Goals for Week 13:
−
#* Upload content from Research Project Proposal & Progress Report onto 2015 group's Wiki page
+
#* Have meeting with Derek to talk about the next task that need to be completed
−
 
+
−
 
+
−
# Week 12 progress:
+
−
#* Calculated the P-value for European languages form Declaration against Somerton code and compare results with 2013 group's.
+
−
#* Calculated chi-squared value between top 20 European languages from declaration against code and top 21 popular European language from Wikipedia against code.
+
−
#* Finishd and submited Semester A proposal and progress report on 5th June.
+
−
# Goals for Week 13:
+
−
#* Finish and upload gourp progress report to Wiki page.
+
−
#* Prepare begin task 3.
+
  
 
=== Semester 1 Week 13===
 
=== Semester 1 Week 13===
Line 191: Line 170:
  
 
# Week 13 progress:
 
# Week 13 progress:
−
#* Uploaded content from Research Project Proposal & Progress Report onto 2015 group's Wiki page
+
#* Exam study, therefore no project work
  
 
# Goals for Exam Period:
 
# Goals for Exam Period:
−
#* Exam Revision
+
#* Do Exam
−
#* Commence Task 2
+
−
 
+
−
 
+
−
# Week 13 progress:
+
−
#* Uploaded research project proposal & progress report onto 2015 group's Wiki page.
+
−
 
+
−
# Goals for Exam Period:
+
−
#* Review for exam.
+
  
 
===Mid Year Break Week 1===
 
===Mid Year Break Week 1===
Line 208: Line 179:
  
 
# Week 1 progress:
 
# Week 1 progress:
−
#* Installed Microsoft Visual Studio.
+
#* Meet up with partner to talk about next task
−
#* Created and ran n-gram test code using quick start guide.
+
#* Ethics been approved
−
 
+
 
# Goals for Week 2:
 
# Goals for Week 2:
−
#* Re-evaluate the use of the Microsoft Database and perhaps consider the Google alternative.
+
#* Start doing DNA task
−
 
+
−
 
+
−
# Week 1 progress:
+
−
#* Finish Chi-squared tests update(If the letter appeared 0 time.
+
−
 
+
−
Use number 1 to instead of 0 for numbers)
+
−
 
+
−
# Goals for Week 2:
+
−
 
+
−
#* Go to Gold Coast.
+
  
 
===Mid Year Break Week 2===
 
===Mid Year Break Week 2===
  
 
# Week 2 progress:
 
# Week 2 progress:
−
#* Upon further research, and consulting Dr Matthew Berryman and the Microsoft Web N-Gram Team, I found that the Microsoft database would require too many calls for use with our application and so the Google alternative must be considered.
+
#* Degrading the DNA is nearly completed
−
#* Enquired about access to the collated database from the University of Pennsylvania Linguistic Data Consortium, was quoted a cost of 150 USD for the license, plus 50 USD postage of the physical DVDs upon which they are stored.
+
#* Matlab code is nearly done
 
    
 
    
 
# Goals for Week 3:
 
# Goals for Week 3:
−
#* Begin writing search engine for use with Google corpus.
+
#* Have the code completed
−
 
+
−
# Week 2 progress:
+
−
#* Travel in Gold Coast.
+
−
 
+
−
# Goals for Week 3:
+
−
#* Review One-time pad work from 2013's group.
+
−
#* Search information about One-time pad.
+
  
 
===Mid Year Break Week 3===
 
===Mid Year Break Week 3===
  
 
# Week 3 progress:
 
# Week 3 progress:
−
#* Sought assistance from Dr Matthew Berryman, who wrote code to download the n-gram files from Google and collate them into a single file.
+
#* Code is completed
−
#* Dr Matthew Berryman advised me to use Python to write the database search engine
+
#* Started degrading the DNA, by removing SNPs
−
#* Began to learn python language
+
 
−
#* Began writing search engine
+
−
#* Had Skype conference with Dr Matthew Berryman
+
−
#* Organised meeting with Professor Abbott for next week
+
−
 
+
 
# Goals for Semester 2 week 1:
 
# Goals for Semester 2 week 1:
−
#* Continue learning python and continue working on search engine
+
#* Meet with Derek and show him our progress
−
#* Meet with Professor Abbott
+
−
 
+
−
# Week 3 progress:
+
−
#* Reviewed One-time pad work from 2013's group.
+
−
#* Searched information about One-time pad.
+
−
#* Organised meeting with Professor Abbott for next week.
+
−
# Goals for Semester 2 Week 1:
+
−
#* Met with Professor Abbott and Nick.
+
  
 
===Semester 2 Week 1===
 
===Semester 2 Week 1===
  
 
# Week 1 progress:
 
# Week 1 progress:
−
#* Met with Professor Abbott on Tuesday 28 July.
+
#* Met with Derek, says our work is going well
−
#* Continued working on search engine code in conjunction with Dr Matthew Berryman.
+
#* Degraded the DNA by 10% intervals and now waiting for results
−
#* Ran code on database for first time.
+
−
#* Continued learning python functions used in code for use in thesis writeup.
+
  
 
# Goals for Week 2:
 
# Goals for Week 2:
−
#* Assist Jikai with Task 3
+
#* After results have come, start to analyse the result and figure what the next steps are
−
#* Continue writing up elements for use in thesis.
+
−
 
+
−
# Week 1 progress:
+
−
 
+
−
#* Met with Professor Abbott and Nick on Tuesday 28 July.
+
−
#* Discussed task 3 with Nick.
+
−
# Goals for Week 2:
+
−
 
+
−
#* Started writing code for task 3 One-time pad.
+
  
 
===Semester 2 Week 2===
 
===Semester 2 Week 2===
  
 
# Week 2 progress:
 
# Week 2 progress:
−
#* Recieved results from first test from Matthew Berryman.
+
#* Another set of degrading data was put into Gedmatch to compare results
−
#* Commenced working on obtaining statistical data from results.
+
#* Met with Derek to inform us on what the next step is, which is to find false positives and false negatives and graph results
−
#* Continued writing up elements for thesis.
+
#* Also got told to look into the heritage stuff on Gedmatch
  
 
# Goals for Week 3:
 
# Goals for Week 3:
−
#* Try to complete statistical results.
+
#* Plot false positives and false negatives
−
#* Email Dr Andrew Allison to book consulting time to discuss our task 1 statistical analysis results
+
#* Start looking into booking the mass spectrometer lab
−
 
+
−
# Week 2 progress:
+
−
#* Try to use Matlab to write code for task 3 One-time pad.
+
−
 
+
−
# Goals for Week 3:
+
−
#* Continue writing code for task 3.
+
  
 
===Semester 2 Week 3===
 
===Semester 2 Week 3===
  
 
# Week 3 progress:
 
# Week 3 progress:
−
#* Emailed Dr Andrew Allison and received no reply.
+
#* Mass Spect booked
−
#* Organised appointment with Professor Abbott for Friday 21st of August.
+
#* Obtained Hair for Mass spect
−
#* Emailed Dr Matthew Berryman about discrepancy between raw results and initial results using count column.
+
  
 
# Goals for Week 4:
 
# Goals for Week 4:
−
#* Meet with Professor Abbott on Friday.
+
#* Complete Mass Spect
−
#* Discuss n-gram search results with Dr Berryman and Professor Abbott to decide whether/what type of statistical analysis is to be conducted.
+
#* Write Thesis
−
 
+
−
# Week 3 progress:
+
−
#* Continued using Matlab to write code for task 3.
+
−
 
+
−
 
+
−
# Goals for Week 4:
+
−
#* Use Matlab to write code for task 3.
+
−
#* Finish case(4M and 2W) Chi-squared value calculations.
+
  
 
===Semester 2 Week 4===
 
===Semester 2 Week 4===
  
 
# Week 4 progress:
 
# Week 4 progress:
−
#* Met with Professor Abbott for Friday 21st of August.
+
#* Mass Spect Completed
−
#* Decided that we are unable to conduct statistical analysis from results of n-gram search, so results to be cleaned up and presented as part of final report.
+
#* Finished Task 3
  
 
# Goals for Week 5:
 
# Goals for Week 5:
−
#* Clean up search results for presentation
+
#* Analysis results from mass spect
−
#* Visit Dr Allison's office to organise a meeting
+
−
#* Conduct verification and testing of Jikai's one-time pad code.
+
−
 
+
−
# Week 4 progress:
+
−
#* Meet with Professor Abbott on Friday 21st of August.
+
−
#* Finish programming code for one time pad.
+
−
 
+
−
# Goals for Week 5:
+
−
#* Add last function to the code.
+
−
#* Review our task 1 statistical analysis.
+
  
 
===Semester 2 Week 5===
 
===Semester 2 Week 5===
  
 
# Week 5 progress:
 
# Week 5 progress:
−
#* Emailed Hong Gunn Chew about University of Wollongong invoicing University of Adelaide for server and processing costs, using project budget.
+
#* Made some progress with the results
−
#* Tried reaching Dr Andrew Allison via his office phone on multiple days but he was not in.
+
#* Met with Derek
−
#* Spoke to Electrical Engineering office and was given an alternate email to try to reach him at.
+
−
#* Emailed Dr Allison to organise meeting.
+
−
#* Verified Jikai's One Time Pad code and suggested changes and improvements.
+
  
 
# Goals for Week 6:
 
# Goals for Week 6:
−
#* Clean up search results for presentation.
+
#* Continue on analyzing results
−
#* Continue writing up n-gram search section of thesis.
+
#* Continue writing thesis  
−
#* Obtain mass spectrometer data from Professor Abbott to begin extension task (task 4).
+
−
 
+
−
# Week 5 progress:
+
−
#* Improve the matlab code and add last function in order to decipher the code by last character.
+
−
#* Review our chi-squared result.
+
−
#* Complete our group wiki page under IEEE home page.
+
−
# Goals for Week 6:
+
−
#* Continue improving code and add function.
+
−
#* Organise a meeting with Dr Allison to discusss our statistic result.
+
  
 
===Semester 2 Week 6===
 
===Semester 2 Week 6===
  
 
# Week 6 progress:
 
# Week 6 progress:
−
#* Met with Dr Andrew Allison and discussed possibility of attempting statistical analysis from task 1 again using Bayes' theorem.
+
#* Made some progress with the results
−
#* Continued writing code to clean up search results for presentation.
+
−
#* Continued writing up n-gram search section of thesis.
+
−
 
+
 
# Goals for Week 7:
 
# Goals for Week 7:
−
#* Arrange meeting with Professor Abbott
+
#* Continue on analyzing results
−
#* Send formal application for funding to Dr Hong-Gunn Chew including amended and forecasted budget.
+
#* Continue writing thesis
−
#* Email Matthew Berryman for advice on proposed method.
+
−
#* Finish clean up code and excel import of data.
+
−
 
+
−
# Week 6 progress:
+
−
#* Continued improving code and add last function.
+
−
#* Met Dr Allison and get some useful suggetions to improve our statistic tests.
+
−
# Goals for Week 7:
+
−
#* Finished one time pad code improvement.
+
−
#* Organise a meeting with Prof Abbott and Dr Matthew to discuss our statistic results.
+
  
 
===Semester 2 Week 7===
 
===Semester 2 Week 7===
  
 
# Week 7 progress:
 
# Week 7 progress:
−
#* Arranged to have meeting with Professor Abbott and Dr Berryman in week 8.
+
#* Found Strontium levels are lower in Adelaide compared to Bali
−
#* Created agenda of items of importance to discuss in meeting.
+
#* Continue analysing other elements to find any significants
−
#* Had email discussion with Dr Berryman and Dr Allison about statistical analysis approach.
+
−
#* Completed cleanup code.
+
−
#* Ran completed cleanup code on search results.
+
−
#* Cleanup code produced repeated results, so commenced writing double remover code.
+
−
#* Completed application for funding to be sent to Dr Hong-Gunn Chew, waiting on invoices for proof of purchases from Dr Berryman.
+
  
 
# Goals for Week 8:
 
# Goals for Week 8:
−
#* Complete and run doubles remover code and import outputs into excel document.
+
#* Continue writing thesis
−
#* Email application for funding to Dr Hong-Gunn Chew.
+
−
#* Meet with Professor Abbott and Dr Berryman to discuss project progress so far and direction of further work.
+
−
 
+
−
# Week 7 progress:
+
−
#* Finished one time pad code improvement.
+
−
#* Organise a meeting with Prof Abbott and Dr Matthew to discuss our statistic results.
+
−
#* The matlab code was able to decode the Somerton code by using different letter locations and last letter as key.
+
−
 
+
−
# Goals for Week 8:
+
−
#* Meet both Professor Abbott and Dr Berryman to discuss project progress so far and we will start task 4 or update our task 1 results in the midbreak.
+
−
#* Use matlab code to output decoding messages and find some meaningful words.
+
  
 
===Semester 2 Week 8===
 
===Semester 2 Week 8===
  
 
# Week 8 progress:
 
# Week 8 progress:
−
#* Created agenda for meeting with Professor Abbott, Dr Berryman and Jikai.
+
#* Graphs made with the different elements
−
#* Met with Professor Abbott, Dr Berryman and Jikai on Friday 18 September.
+
 
−
#* In meeting, decided upon using texts from project Gutenberg to redo chi-squared statistical analysis on the language of origin of the code to overcome the issue of letters appearing 0 times in using the UN Declaration of Human Rights as the base text.
+
−
#* Received invoices from Dr Berryman and emailed completed application for funding to Dr Hong-Gunn Chew.
+
−
#* Completed doubles remover code.
+
  
 
# Goals for Mid-Semester Break:
 
# Goals for Mid-Semester Break:
−
#* Process results using doubles remover code and import into excel.
+
#* Continue writing thesis
−
#* Obtain useful results from n-gram search for use in final report.
+
#* Start making the poster
−
#* Submit exhibition requirements.
+
−
#* Fix up all original chi-squared test results.
+
−
#* Perform Chi-squared reanalysis using project Gutenberg texts.
+
  
−
# Week 8 progress:
 
−
#* Met Professor Abbott and Dr Berryman on Friday 18/09/2015.
 
−
#* In the meeting, decided we should redo our task 1 by using texts from project Gutenberg instead of using the UN Declaration of Human Rights as the base text.
 
−
#* Used matlab code to output decoding messages and try to find some meaningful words.
 
  
−
 
−
# Goals for Mid-Semester Break:
 
−
#* Redo our chi-squared test on task 1.
 
−
#* Search text files on project Gutenberg webpage and format them.
 
−
#* Analyse results in task 3.
 
  
 
===Mid Semester Break===
 
===Mid Semester Break===
 
# Week Mid-Semester Break progress:
 
# Week Mid-Semester Break progress:
−
#* Submitted Exhibition requirements and created shortened project title and project description for display.
+
#* Poster nearly completed
−
#* Extended doubles remover code to work for results that appear more than twice.
+
−
#* Processed results and output final results in alphabetical order for comparison with raw results and cleaned up results.
+
−
#* Also processed results and output in order of highest number of year occurrence to lowest.
+
−
#* Tried importing results into Excel but ran out of rows, as data was too large, so instead exported full results for use in final dump of work, as well as top 20 results ranked in order of occurrence for each group of n-grams from 1-5 for inclusion in appendices of final report/thesis/wiki.
+
−
#* Discussed finding interesting/useful combinations of 5 grams for presenting in final report with Matthew Berryman
+
−
#* Received most popular interesting/useful 5 gram combinations from Dr. Berryman.
+
−
#* Fixed up task 1 initial results with count changed from 0 to 1, fixed up second refinement, for letters with 0 count, changing frequency to 0.001, and commenced re-analysis of some of the top 20 European languages with larger text files from project Gutenberg to allow no letters with count 0 to occur to verify our top 20 European results.
+
−
#* Wrote instructions on how to use initial and all letter counter java files from 2013 group’s work
+
−
 
+
−
# Goals for week 9:
+
−
#* Organise meeting and meet with Professor Abbott
+
−
#* Visit Maths Learning Centre to see if there is any way to obtain a p-value from our new data sets.
+
−
#* Finalise all results, figures, graphs etc. for use in presentation and thesis/final report
+
−
#* Commence creating slideshow for final presentation
+
−
#* Commence writing thesis skeleton for use in final presentation
+
−
 
+
−
====Jikai====
+
−
# Week Mid-Semester Break progress:
+
−
#* Redo our chi-squared test on task 1.
+
−
#* Searched text files on project Gutenberg webpage and format them. The new text files from project Gutenberg allows no letters with count 0 and we uesd them to verify our chi-squared test results.
+
−
#* Analyse results in task 3, pick some of them to used in our Final Seminar and Final Report.
+
  
 
# Goals for week 9:
 
# Goals for week 9:
−
#* Meet Professor Abbott and Dr Berryman to discuss our update results for task 1.
+
#* Continue writing thesis
−
#* Start writing slides for our Final Seminar presentation.
+
#* Complete poster so Derek can draft
−
#* Start writing our Final thesis.
+
  
 
===Semester 2 Week 9===
 
===Semester 2 Week 9===
 
# Week 9 progress:
 
# Week 9 progress:
−
#* Obtained promising results from re-analysis of a portion of the top 20 European languages using project Gutenberg text files and the chi-squared method.
+
#* Poster completed, sent to get drafted
−
#* Organised meeting and met with Professor Abbott and Dr. Berryman on Thursday 8 October at 11am.
+
#* Thesis needs to be completed, so it can be drafted
−
#* Conclusions from meeting were to increase the sample size of the English portion of our Project Gutenberg corpus to 20 concatenated novels, and to take 100 44 letter samples from this to obtain our average chi-squared value for English to calculate our English p-value significance level.
+
−
#* Visited Maths Learning Centre and was given some useful advice on how to compare our control p-values vs code p-values.
+
−
#* Attempted normal distribution p-value analysis method on chi-squared value of code versus average chi-squared value of English samples but did not work since lower chi-squared values are better, not closer chi-squared values.
+
−
#* Finalised all results, figures, graphs etc. for use in presentation and thesis/final report.
+
−
#* Commenced creating slideshow for final presentation.
+
−
#* Continued writing thesis for inclusion of task 4: statistical frequency of letters reanalysis in final presentation.
+
  
 
# Goals for week 10:
 
# Goals for week 10:
−
#* Complete slideshow and talk for final presentation.
+
#* Complete thesis
−
#* Present final presentation on Tuesday 13 October at 5pm.
+
#* Edit poster after it is drafted
−
#* Write python code in order to increase the sample size of the English portion of our Project Gutenberg corpus to 20 concatenated novels, and to take 100 44 letter samples from this.
+
#* Start the final seminar slides
−
#* Obtain average chi-squared value for English to calculate our English p-value significance level.
+
−
#* Obtain chi-squared and p-value for new English base text vs Somerton Man code.
+
−
#* Continue writing thesis.
+
−
 
+
−
# Week 9 progress:
+
−
#* Meet Professor Abbott and Dr Berryman on Thursday 08/10/2015 and discuss our update results on task 1.
+
−
#* Discuss with Nick about the Final Seminar workplan and time.
+
−
#* Start writing slides for our Final Seminar presentation.
+
−
 
+
−
 
+
−
# Goals for week 10:
+
−
#* The final seminar will hold on Tuesday 13/10/2015.
+
−
#* Continue updating and improving our Chi-squared test for task 1.
+
−
#* Start writing our Final thesis.
+
  
 
===Semester 2 Week 10===
 
===Semester 2 Week 10===
 
# Week 10 progress:
 
# Week 10 progress:
−
#* Completed slideshow and talk for final presentation.
+
#* Poster completed and submitted
−
#* Presented final presentation on Tuesday 13 October at 5pm.
+
#* Final seminar slides done and sent to Derek to get drafted
−
#* Completed python codes for implementing increased sample size reanalysis for Task 4
+
#* Sent Derek our thesis to get drafted
−
#* Obtained average chi-squared value for English to calculate our English p-value significance level.
+
−
#* Obtained chi-squared and p-value for new English base text vs Somerton Man code.
+
−
#* Continued writing thesis.
+
  
 
# Goals for week 11:
 
# Goals for week 11:
−
#* Complete and submit final report/honours thesis
+
#* Edit final seminar slides when drafted
−
#* Complete honours poster
+
#* Edit thesis
−
 
+
#* Prepare for expo
−
# Week 10 progress:
+
−
#* Finished the PPT slides for final seminar.
+
−
#* Presented The final seminar on Tuesday 13/10/2015.
+
−
#* Downloaded and formatted English novels from Project Gutenberg, and sent results to Nick.
+
−
#* Started writing our Final thesis.
+
−
.
+
−
 
+
−
# Goals for week 11:
+
−
#* Finish our Final thesis in Wiki page.
+
  
 
===Semester 2 Week 11===
 
===Semester 2 Week 11===
 
# Week 11 progress:
 
# Week 11 progress:
−
#* Completed and submitted Final Report/Honours Thesis
+
#* Get Thesis Drafted
−
#* Designed and submitted project exhibition poster
+
#* Final seminar slides submitted
−
#* Organised project exhibition resources
+
#* Expo completed
  
 
# Goals for week 12:
 
# Goals for week 12:
−
#* Present at project exhibition
+
#* Finish Thesis
−
 
+
#* Prepare for final seminar
−
# Week 11 progress:
+
#* Give all relevant files to Derek
−
*# Finished final thesis before Wednesday 21/10/2015.
+
−
*# Prepared for exhibition next week.
+
−
 
+
−
 
+
−
# Goals for week 12:
+
−
*# Exhibition will hold between Monday (26/10/2015) and Tuesday (27/10/2015).
+
−
*#
+
  
 
===Semester 2 Week 12===
 
===Semester 2 Week 12===
 
# Week 12 progress:
 
# Week 12 progress:
−
#* Presented at Ingenuity Exhibition
+
#* Thesis done and submitted
−
#* Collated work for final dump to be submitted Friday 6 November
+
#* Work on Final Seminar
−
#* Organised final meeting with Derek for Friday 6 November
+
  
−
# Week 12 progress:
+
# Goals for week 13:
−
#*  
+
#* Prepare for final seminar
−
#*
+
 
−
#*  
+
===Semester 2 Week 13===
−
#*
+
# Week 13 progress:
 +
#* Final seminar completed
 +
#* Closed project
  
 
==See also==
 
==See also==
−
* [[Cipher Cracking 2013]]
+
 
−
* [[Semester A Progress Report 2013 - Cipher cracking]]
+
−
* [[Semester B Final Report 2013]]
+
−
* [[Cipher Cross-off List]]
+
  
 
==Back==
 
==Back==

Latest revision as of 11:35, 3 November 2018

Weekly progress

Semester 1 Week 1

  1. Week 1 progress:
    • Organise meeting with Prof. Abbott.
    • Contacted project partner and had a meeting.
    • Attended the literature training session with project partner at Bar Smith library.
  1. Goals for Week 2:
    • Meet Prof. Abbott.
    • Understand the aims of the project

Semester 1 Week 2

  1. Week 2 progress:
    • Met with Prof. Abbott.
    • Conducted Background research by reading about previous final year projects.
  1. Goals for Week 3:
    • Start preparing for the Proposal Seminar.
    • Research about Somertons Mans letters.

Semester 1 Week 3

  1. Week 3 progress:
    • Start Work Break Down Structure
    • Commenced work on project Gantt Chart
    • Organised meeting with Professor Abbott
    • Worked on Proposal
    • Met with project partner
  1. Goals for Week 4:
    • Attend meeting with Professor Abbott
    • Finish project Gantt Chart

Semester 1 Week 4

  1. Week 4 progress:
    • Finished project Gantt Chart
    • Met with Professor Abbott
    • Worked on Proposal
    • Met with project partner
    • MATLAB coding started for Task 1
  1. Goals for Week 5:
    • Finish Power point for proposal
    • Make more progress in with Task 1

Semester 1 Week 5

  1. Week 5 progress:
    • Power Point nearly completed
    • Names of horses, city names etc. have been finalized
    • Task 1 about 50% complete
  1. Goals for Mid-Semester Break:
    • Power Point has to be completed
    • Prepare speech for proposal
    • Meet with project partner in break to complete Task 1

Semester 1 Mid-Semester Break

  1. Mid-Semester Break progress:
    • Met with project partner
    • More coding done for Task 1
    • Graphs have been produced for Task 1
    • Task 1 75% complete
    • Power Point got drafted by Prof. Abbott
    • Proposal seminar completed
    • DNA testing kit has been delivered
  1. Goals for Week 6:
    • Organise meeting with Prof. Abbott
    • Complete Task 1
    • Collect the DNA kit
    • Send DNA kit ASAP

Semester 1 Week 6

  1. Week 6 progress:
    • Sent DNA to 23andMe
    • Had meeting with Derek
    • An extension of Task 1 was given, to further look into the book
    • Still working on Task 1
  1. Goals for Week 7:
    • Work on the extension task
    • Start working on thesis

Semester 1 Week 7

  1. Week 7 progress:
    • Original of Task 1 is completed
    • Extension of Task 1 near completion
    • The code is complete to count the letter in each line of the book
    • Thesis layout completed
    • Thesis introduction started
  1. Goals for Week 8:
    • Complete extension of Task 1
    • Complete Thesis introduction
    • Oragnize meeting with Derek

Semester 1 Week 8

  1. Week 8 progress:
    • Meeting with Derek
    • Derek said that we have good progress
    • Meeting with partner, to talk about thesis
    • Task 1 is fully completed
    • Results show that horse names are not the mysterious code
  1. Goals for Week 9:
    • Work on thesis

Semester 1 Week 9

  1. Week 9 progress:
    • DNA results have come back, need to have meeting with Derek ASAP
    • Thesis introduction completed
    • Sent drafts to each other
  1. Goals for Week 10:
    • Have meeting with Derek
    • Continue working on the thesis

Semester 1 Week 10

  1. Week 10 progress:
    • Meeting with Derek
    • Informed that we can't do any of the DNA task until the ethics is signed
    • Showed Derek results for task 1 and told us it was good
    • Drafted each others thesis and sent it back to each other
  1. Goals for Week 11:
    • Only task to do now is the thesis

Semester 1 Week 11

  1. Week 11 progress:
    • Thesis introduction completed again
    • Thesis technical background started
    • Thesis project management started
  1. Goals for Week 12:
    • Have the thesis completed by Tuesday, so we can send our drafts to each other

Semester 1 Week 12

  1. Week 12 progress:
    • Thesis completed and sent to partner
    • Draft each others thesis
    • Final copy of draft thesis completed and sent to my uni
  1. Goals for Week 13:
    • Have meeting with Derek to talk about the next task that need to be completed

Semester 1 Week 13

  1. Week 13 progress:
    • Exam study, therefore no project work
  1. Goals for Exam Period:
    • Do Exam

Mid Year Break Week 1

  1. Week 1 progress:
    • Meet up with partner to talk about next task
    • Ethics been approved
  1. Goals for Week 2:
    • Start doing DNA task

Mid Year Break Week 2

  1. Week 2 progress:
    • Degrading the DNA is nearly completed
    • Matlab code is nearly done
  1. Goals for Week 3:
    • Have the code completed

Mid Year Break Week 3

  1. Week 3 progress:
    • Code is completed
    • Started degrading the DNA, by removing SNPs
  1. Goals for Semester 2 week 1:
    • Meet with Derek and show him our progress

Semester 2 Week 1

  1. Week 1 progress:
    • Met with Derek, says our work is going well
    • Degraded the DNA by 10% intervals and now waiting for results
  1. Goals for Week 2:
    • After results have come, start to analyse the result and figure what the next steps are

Semester 2 Week 2

  1. Week 2 progress:
    • Another set of degrading data was put into Gedmatch to compare results
    • Met with Derek to inform us on what the next step is, which is to find false positives and false negatives and graph results
    • Also got told to look into the heritage stuff on Gedmatch
  1. Goals for Week 3:
    • Plot false positives and false negatives
    • Start looking into booking the mass spectrometer lab

Semester 2 Week 3

  1. Week 3 progress:
    • Mass Spect booked
    • Obtained Hair for Mass spect
  1. Goals for Week 4:
    • Complete Mass Spect
    • Write Thesis

Semester 2 Week 4

  1. Week 4 progress:
    • Mass Spect Completed
    • Finished Task 3
  1. Goals for Week 5:
    • Analysis results from mass spect

Semester 2 Week 5

  1. Week 5 progress:
    • Made some progress with the results
    • Met with Derek
  1. Goals for Week 6:
    • Continue on analyzing results
    • Continue writing thesis

Semester 2 Week 6

  1. Week 6 progress:
    • Made some progress with the results
  2. Goals for Week 7:
    • Continue on analyzing results
    • Continue writing thesis

Semester 2 Week 7

  1. Week 7 progress:
    • Found Strontium levels are lower in Adelaide compared to Bali
    • Continue analysing other elements to find any significants
  1. Goals for Week 8:
    • Continue writing thesis

Semester 2 Week 8

  1. Week 8 progress:
    • Graphs made with the different elements


  1. Goals for Mid-Semester Break:
    • Continue writing thesis
    • Start making the poster


Mid Semester Break

  1. Week Mid-Semester Break progress:
    • Poster nearly completed
  1. Goals for week 9:
    • Continue writing thesis
    • Complete poster so Derek can draft

Semester 2 Week 9

  1. Week 9 progress:
    • Poster completed, sent to get drafted
    • Thesis needs to be completed, so it can be drafted
  1. Goals for week 10:
    • Complete thesis
    • Edit poster after it is drafted
    • Start the final seminar slides

Semester 2 Week 10

  1. Week 10 progress:
    • Poster completed and submitted
    • Final seminar slides done and sent to Derek to get drafted
    • Sent Derek our thesis to get drafted
  1. Goals for week 11:
    • Edit final seminar slides when drafted
    • Edit thesis
    • Prepare for expo

Semester 2 Week 11

  1. Week 11 progress:
    • Get Thesis Drafted
    • Final seminar slides submitted
    • Expo completed
  1. Goals for week 12:
    • Finish Thesis
    • Prepare for final seminar
    • Give all relevant files to Derek

Semester 2 Week 12

  1. Week 12 progress:
    • Thesis done and submitted
    • Work on Final Seminar
  1. Goals for week 13:
    • Prepare for final seminar

Semester 2 Week 13

  1. Week 13 progress:
    • Final seminar completed
    • Closed project

See also

Back

Retrieved from "https://www1.eleceng.adelaide.edu.au/personal/dabbott/wiki/index.php?title=Cipher_cracking_2018_weekly_progress&oldid=21187"