Difference between revisions of "Cipher cracking 2011 weekly progress"

From Derek
Jump to: navigation, search
(→‎Patrick)
(→‎Semester 2 Week 12)
 
(41 intermediate revisions by 2 users not shown)
Line 617: Line 617:
 
====Steven====
 
====Steven====
 
# Week 8 work:
 
# Week 8 work:
−
#* Work for Cipher GUI:
+
#* Work for Cipher GUI:  
−
#* Work for web crawler/pattern matcher:
+
#** Implemented the Double Playfair Cipher including interactive key functionality in accordance with request from Derek after previous meeting
 +
#** Added interactive key functionality for Hill Cipher – all ciphers in Cipher GUI now have interactive key functionality!
 +
#* Work for web crawler/pattern matcher:  
 +
#** Redesigned web crawler GUI to include:
 +
#*** Tab layout to separate seed, pattern and saving controls
 +
#*** Ability to add/edit/remove multiple search seeds
 +
#*** Capacity to add/edit/remove multiple search patterns
 +
#*** Control over number of pages to search
 +
#*** Control over search depth
 +
#** Began integrating the web crawler/pattern matcher into the GUI frontend
 +
#** Examined resource highlighted by Matthew – search engine company blekko allowing public submission of regular expressions to analyse
 
#* Management and documentation:
 
#* Management and documentation:
 +
#** Continued final report (to be submitted 21/10/11)
 +
#** Decided with Patrick to postpone commencement of final seminar until next week to better accommodate advice from the scheduled meeting
 
# Goals for week 9:
 
# Goals for week 9:
 
#* Meet with Derek and Patrick to discuss progress and final seminar
 
#* Meet with Derek and Patrick to discuss progress and final seminar
 +
#* Commence work on presentation for final seminar to be held 30/09/11
 +
#* Upload investigated ciphers to cross-off list (not uploaded earlier as I had no access to project page)
 +
#* Create demonstration web crawler/pattern matcher GUI
 +
#* Formulate some regular expressions with Patrick that can be submitted to the blekko analysis tool
 +
#* Update information panels in Cipher GUI
  
 
====Patrick====
 
====Patrick====
 
# Work Done this Week
 
# Work Done this Week
 +
#* Examined the [[Cipher Cross-off List#Playfair_Cipher|Playfair Cipher]] using the code and the Cipher GUI updated last week. Concluded we can rule it out. [[Cipher Cross-off List#Playfair_Cipher|Click here for reasoning.]]
 +
#* Cipher GUI work:
 +
#** Fixed a bug in Playfair Cipher relating to forming the Playfair Square
 +
#** Changed the default plaintext display to lowercase and ciphertext display to uppercase as per cryptological norms.
 +
#* Web Crawler/Pattern Matcher work:
 +
#** Completely re-designed  and re-coded pattern matching code for the following reasons:
 +
#*** Make the setup neater, easier to understand and more flexible to change/add functionality.
 +
#*** New understanding on how the web crawler provides data enabled a more direct design approach.
 +
#** Created a demo pattern matching web crawler that successfully detected patterns in a live Web-Crawling test!
 +
#** Had a look at an article, pointed out by Matthew, regarding a similar project by the search engine company ''blekko'' that allows users to submit a regex to run a search on.
 
# Goals for Next Week
 
# Goals for Next Week
 +
#* Further investigation of ciphers for the Cipher Cross-off List
 +
#* Implement a cipher on the Cipher GUI
 +
#* Work with Steve in integrating the web crawler/pattern matcher into a GUI
 +
#* Upgrade the pattern matching code further
 +
#* Come up with some regex's we can submit to the ''blekko'' crawler to see if we can get them to test for us
 +
#* Meet with Steve and Derek regarding progress to date and plans for the Final Seminar
 +
#* Begin work on said Final Seminar.
  
 
=== Semester 2 Week 9===
 
=== Semester 2 Week 9===
 
====Steven====
 
====Steven====
 
# Week 9 work:
 
# Week 9 work:
 +
#* Work for Cipher GUI:
 +
#** Modification of Cipher GUI postponed to ensure there are no conflicting versions with the updates Patrick plans to make
 +
#* Work for web crawler/pattern matcher:
 +
#** Met with Patrick and reviewed the web crawler GUI
 +
#** Continued integrating the web crawler and pattern matching software into the GUI frontend
 +
#** Reconfigured web crawler and pattern matching programs for improved user operation and feedback
 +
#** Added the ability to select exact and initial pattern definitions into the web crawler GUI
 +
#** Reviewed the blekko analysis tool highlighted by Matthew and concluded a submission would not be made. The major factors were the inability to control the web crawl and uncertainty as to whether our submission would be used.
 +
#** Formulated regular expressions initially for use with blekko analysis tool. Now will be used with the additional pattern matching functionality that Patrick is developing.
 +
#* Management and documentation:
 +
#** Attended a meeting with Derek and Patrick and summarised the progress to date. Plans for the final seminar were also discussed.
 +
#** Commenced work on presentation for final seminar (30/09/11)
 
# Goals for "break":
 
# Goals for "break":
 +
#* Week 1 (final seminar 30/09/11)
 +
#** Continue compilation of final seminar presentation
 +
#** Present the final seminar on Friday 30/09/11
 +
#** Complete integration of web crawler, pattern matcher and graphical user interface
 +
#** Update web crawler graphical user interface to include regular expression functionality
 +
#** Any further software modifications that are required for the demonstration component of the final seminar
 +
#* Week 2 (final seminar complete)
 +
#** Meet with Patrick and review all work required for project closeout
 +
#** Begin poster(s) for the final exhibition (28/10/11)
 +
#** Plan project video – acquire all additional resources
 +
#** Work with Patrick to establish how our software can be deployed on the project wiki
 +
#** Continue final report (submission 21/10/11)
  
 
====Patrick====
 
====Patrick====
 
# Work Done this Week
 
# Work Done this Week
 +
#* Cipher Cross-off List
 +
#** Ruled out Flat Frequency Response ciphers from being used in creating the code - done via statistics. Proof is [[Cipher_Cross-off_List#Flat Frequency Ciphers|here.]]
 +
#*** This rules out both the [[Cipher_Cross-off_List#One Time Pad|One Time Pad]] and [[Cipher_Cross-off_List#Homophonic Substitution Ciphers|Homophonic Substitution Ciphers.]]
 +
#** Identified what sort of Keyword could have been used if the [[Cipher_Cross-off_List#Keyword Cipher|Keyword Cipher]] was used to create the code.
 
#* Cipher GUI work:
 
#* Cipher GUI work:
 +
#** Added Substitution Cipher that lets the user pick their own Cipher Alphabet.
 +
#** Improved Keyword Cipher by added a JTable that displays the current Cipher Alphabet according to the user-entered keyword.
 +
#** Improved Playfair Cipher by addinga Table that shows the Playfair square. The table updates when the Keyword is changed by the user.
 +
#** Added the unique center panel for the Caesar Cipher.
 
#* Web Crawler/Pattern Matcher work:
 
#* Web Crawler/Pattern Matcher work:
 +
#** Discussed the Crawler GUI setup with Steve who is working on the integration into the GUI
 +
#** Investigated Regular Expressions (regexs) for the purpose of using them for a crawl on ''blekko's'' web data as identified last week
 +
#*** Identified Regexs that could successfully match to Initial Letters and character patterns
 +
#*** Decided that ''blekko'' wouldn't be much help since they aren't providing the option to run a web crawl, they are just offering to run two crawls a day themselves.
 +
#*** However these regexs have given me an idea and the production of an additional Pattern Matching feature is under way! A Regex option - crawl the web for a regex of your choice.
 +
#* Project Management work:
 +
#** Had a meeting with Steve and Derek. Addressed progress on work to date, and discussed plans for the final seminar coming up.
 +
#** Began planning on the content of the Final Seminar.
 
# Goals for Next Fortnight
 
# Goals for Next Fortnight
 +
#* Week 1
 +
#** Work on the Final Seminar
 +
#** More work on the Final Seminar
 +
#** Even More work on the Final Seminar! Present the Seminar on Friday.
 +
#** Look into recording our Seminar so Derek (who is overseas) can watch it.
 +
#** Implement a Regular Expression Pattern Matching option in the web crawler.
 +
#** Do whatever other final touches are needed for the Web Crawler/Pattern Matcher and Cipher GUI for the Final Seminar's Demonstration section.
 +
#* Week 2 (Seminar over)
 +
#** Meet with Steven to identify work still needed to finish off the project - review progress on the Gantt Chart
 +
#** Begin work on the Poster(s) for the Project Exhibition.
 +
#** Begin planning our project Video
 +
#** Look into how we can host our software on the internet
  
 
=== Semester 2 Mid-Semester Break===
 
=== Semester 2 Mid-Semester Break===
 
====Steven====
 
====Steven====
 
# Mid-semester break work:
 
# Mid-semester break work:
 +
#* Week 1:
 +
#** Significant work conducted on presentation for final seminar – rehearsals conducted over three days
 +
#** Presented final seminar (30/09/11)
 +
#** Modified web crawler and pattern matcher for demonstration with final seminar
 +
#** Integrated the web crawler, pattern matcher and GUI frontend into a single piece of software
 +
#** Added regular expression functionality to web crawler graphical user interface
 +
#** Downloaded videos for possible use in project video
 +
#* Week 2:
 +
#** Project video:
 +
#*** Analysed 2009 and 2010 project videos.
 +
#*** Commenced planning of project video.
 +
#*** Acquired more potential videos for use in the project video.
 +
#** Project poster:
 +
#*** Examined 2009 and 2010 poster designs
 +
#*** Commenced formulation of a layout for the project poster.
 +
#*** Researched possible content for additional poster(s).
 +
#** Reviewed and updated the cipher cross-off list.
 +
#** Continued final report (to be submitted 21/10/11)
 
# Goals for week 10:
 
# Goals for week 10:
 +
#* Attend project exhibition briefing with Braden Phillips (11/10/11)
 +
#* Meet with Patrick to discuss requirements for project closeout
 +
#* Add webpage prefix filtering capability to the web crawler
 +
#* Continue poster(s) for the final exhibition
 +
#* Continue final report
 +
#* Complete conversion and upload of final seminar video onto YouTube.
  
 
====Patrick====
 
====Patrick====
 
# Work Done this Fortnight
 
# Work Done this Fortnight
 +
#* First Week
 +
#** Plenty of work on Final Seminar; for a "Holidays week" in at uni each day but Monday!
 +
#** Gave Final Seminar; was well received
 +
#** Downloaded software to enable recording of Final Seminar for Derek who is overseas.
 +
#** Implemented Regular Expression Pattern Matching into the Web Crawler!
 +
#** Fixed+Modified other issues with Web Crawler/Pattern Matcher in preparation for Final Seminar demonstration.
 +
#* Second Week
 +
#** Poster:
 +
#*** Did background research for our project poster, including examining multiple previous year examples
 +
#*** Identified some possible setups for our main project poster
 +
#*** Identified possible content for a second poster
 +
#** Video:
 +
#*** Did background research for the planned video including analysing previous years' videos
 +
#*** Began planning some content ideas for our video
 +
#** Hosting Software:
 +
#*** Successfully packaged the current Pattern Crawler into a single executable jar file
 +
#*** A lot of work integrating the CipherGUI into a single jar file, including its images. Successfully integrated all but two ciphers so far.
 +
#*** Looked into hosting software on the wiki page. Due to limited acceptable Upload file types, they will most likely have to be uploaded as jar files in a zipped package.
 
# Goals for Next Week
 
# Goals for Next Week
 +
#* Finish off enabling the CipherGUI to be compiled into a single executable jar file
 +
#* Help Steven with numerous bug fixes and required functionality implementations in the Pattern Crawler...
 +
#* Design pattern matching code to enable a search for multiple patterns on the same page - e.g. to return a result both patterns must be present.
 +
#* Finish compiling/packaging/editing the Final Seminar video and host it on YouTube so others (inc. Derek and Matthew) can view it.
 +
#* Attend lecture on Exhibition to be held Tuesday
 +
#* Continue work on Posters for Exhibition
 +
#* Continue work on project Video
 +
#* Begin work on Final Report; due end Week 11.
  
 
=== Semester 2 Week 10===
 
=== Semester 2 Week 10===
 
====Steven====
 
====Steven====
 
# Week 10 work:
 
# Week 10 work:
 +
#* Attended project exhibition briefing with Braden Phillips
 +
#** Clarified timetable of day (28/10/11) and requirements for poster(s)
 +
#* Met with Patrick and discussed project posters, project exhibition, project video and deployment of software on the project wiki
 +
#** Decided on two posters and their layouts for project exhibition.
 +
#* Modified web crawler and pattern matcher to include webpage filtering.
 +
#** A webpage prefix can now be specified in interface to control crawling behaviour.
 +
#* Continued final report (to be submitted 21/10/11)
 +
#* Formatted video of the final seminar into three parts and converted each to a type suitable for YouTube
 
# Goals for week 11:
 
# Goals for week 11:
 +
#* Meet with Derek and Patrick to discuss progress, final seminar video, final report and the final exhibition
 +
#* Attend School of Electrical and Electronic Engineering final year photograph (27/10/11)
 +
#* Complete and submit Final Report
 +
#* Continue posters for the final exhibition (28/10/11)
 +
#* Continue project video
  
 
====Patrick====
 
====Patrick====
 
# Work Done this Week
 
# Work Done this Week
 +
#* Added in the Gronsfeld Cipher's unique description panel in the CipherGUI.
 +
#* Fixed pattern matching bugs in pattern crawler code
 +
#* Modified the layout of some pattern matcher code to make it uniform with other implementations
 +
#* Successfully set up CipherGUI for executable jar file format
 +
#* Met with Steven to discuss poster/video layouts/content
 +
#* Attended Exhibition lecture; got a pep talk from Braden about what is expected
 +
#* Worked with Steve on Final Seminar video and uploaded them to YouTube
 +
#* Final Report commenced
 
# Goals for Next Week
 
# Goals for Next Week
 +
#* Meet with Derek and Steven to discuss project progress to date and closeout
 +
#* Finish and submit Final Report
 +
#* Run pattern matching tests on code segments using complete web crawler.
 +
#* Aid Steve with work on posters/video
  
 
=== Semester 2 Week 11===
 
=== Semester 2 Week 11===
 
====Steven====
 
====Steven====
 
# Week 11 work:
 
# Week 11 work:
 +
#* Week committed to the final report. Submitted to myUni Friday night.
 +
#* Reformatted and compiled the final report onto wiki page.
 +
#* Attended a meeting with Derek and Patrick and summarised current project progress, project closeout, the final report and the final seminar including the project video and poster.
 +
#* Attended School of Electrical and Electronic Engineering final year photograph.
 +
#* Unable to make any further progress with posters or video this week – focus next week.
 
# Goals for week 12:
 
# Goals for week 12:
 +
#* Complete project poster for the project exhibition – a “theme” is yet to be decided.
 +
#* Complete “general interest” poster. Provides information about three other famous ciphers – Zodiac, Beale and Kryptos.
 +
#* Complete project video for use in the project exhibition
 +
#* Attend project exhibition on Friday 28/10/11
 +
#* Assist Patrick to add executable jar files of the CipherGUI and the web crawler/pattern matcher to the project wiki webpage.
 +
#* Provide Derek with a disc of all project software and electronic files.
  
 
====Patrick====
 
====Patrick====
 
# Work Done this Week
 
# Work Done this Week
 +
#* Many an hour on Final Report, submitted Friday night.
 +
#* Many more hours spent reformatting [[Final Report 2011|Final Report]] and compiling onto the wiki page
 +
#* Worked on cleanup and modification of code for both CipherGUI and Web Crawler for Final Report purposes
 +
#* Ran pattern searches on Somerton Code portions using Web Crawler
 +
#* Met with Steven and Derek. Discussed project outcomes, final work to complete, Final Report and video/poster ideas
 
# Goals for Next Week
 
# Goals for Next Week
 +
#* Finish off CipherGUI - namely Trifid/Bifid descriptions and interactive tables
 +
#* Attempt to provide proxy options for Web Crawler to enable crawling on the University network
 +
#* Aid Steve with Poster for Exhibition
 +
#* Aid Steve with Video production
 +
#* Attend Exhibition on Friday 28/10/2011
 +
#* Upload CipherGUI and Web Crawler executable jar files onto project wiki
 +
#* Handover all project products and work to Derek
  
 
=== Semester 2 Week 12===
 
=== Semester 2 Week 12===
 
====Steven====
 
====Steven====
 
# Week 12 work:
 
# Week 12 work:
 +
#* Web crawler/pattern matcher:
 +
#** Added proxy interface fields to the web crawler graphical user interface.
 +
#** Assisted Patrick in the investigation of the proxy settings for crawling on the uni network.
 +
#** Created two alternative executable jar files of Crawler
 +
#*** One that is a single jar file containing all required libraries. Too large to add to the project webpage.
 +
#*** A second split version that segments external libraries from the operational functionality. Will be added to the wiki for public access.
 +
#* CipherGUI:
 +
#** Tested feasibility of compiling CipherGUI into single executable jar file.
 +
#** Created single executable CipherGUI jar file as insurance. Not required - Patrick's version to be uploaded to wiki.
 +
#* Project Exhibition:
 +
#** Worked with Patrick to compile a project poster with the theme of a 40’s era investigation wall.
 +
#** Worked with Patrick to complete second “general interest” poster containing three other famous ciphers – Zodiac, Beale and Kryptos. Uses the same theme as the project poster.
 +
#** Attended Project Exhibition on Friday 28/10/11. Well received with a lot of interest. Unfortunately no prizes.
 +
#* Completed project video. Will be added to the final seminar videos on YouTube and a link from the wiki provided.
 +
#* Management and documentation:
 +
#** Attended a final meeting with Derek and Patrick to conclude project closeout and handover a CD containing all project software and electronic files for the year.
 +
#** Exams next week. Thank you to both Derek and Patrick for all your help throughout the year and a great project.
  
 
====Patrick====
 
====Patrick====
 
# Work Done this Week
 
# Work Done this Week
 +
#* CipherGUI work:
 +
#** Added descriptions and interactive tables to both Bifid and Trifid ciphers
 +
#** Re-coloured all cipher panels for consistency and re-organised central class naming setup of CipherGUI
 +
#** Wrote up README.txt file instructing use of CipherGUI
 +
#** Created final finished product executable jar file of GUI
 +
#** Uploaded CipherGUI onto project wiki-site available here: [[Media:Cipher GUI.rar|Cipher GUI]]
 +
#* Web Crawler work:
 +
#** Investigated proxy settings for crawling on the uni network
 +
#** Successfully identified correct settings (for uni) and designed a way to allow the user to input these into the GUI
 +
#** Wrote up README.txt files - instructions for others to use our crawler.
 +
#** Created final product executable jar file of Crawler
 +
#** Uploaded Crawler jar file onto project wiki-page, see [[Web Crawler 2011]]
 +
#* Exhibition
 +
#** Aided Steven with Poster designs and contents for the main project poster
 +
#** Wrote up Beale story and Zodiac Killer story for the Additional poster
 +
#** Got Additional Poster printed at Officeworks, nice and shiny on glossy paper!
 +
#** Attended exhibition and showed off products to plenty of interest
 +
#* Other:
 +
#** Uploaded project video to YouTube
 +
#** Slept not much
 +
#** Had final meeting with Derek and Steve and handed over CD containing all my work throughout the year.
 +
# Goals for Next Week
 +
#* Start studying for exams!
 +
#* Over and out. Thanks for the brilliant experience this project has been.
  
 
==See also==
 
==See also==
Line 678: Line 903:
 
* [[Stage 2 Progress Report 2011]]
 
* [[Stage 2 Progress Report 2011]]
 
* [[Cipher Cross-off List]]
 
* [[Cipher Cross-off List]]
 +
* [[Media:Cipher GUI.rar|Cipher GUI 2011]]
 +
* [[Web Crawler 2011]]
 +
* [http://www.youtube.com/watch?v=K0DEF-FDLso 2011 YouTube Project Video]
  
 
==Back==
 
==Back==

Latest revision as of 08:24, 9 November 2011

Weekly progress and questions

This is where you record your progress and ask questions. Make sure you update this every week. The deadline is every Friday evening. However, if you sometimes slip a little into the weekend (so long as you don't do it too often) we won't be too hard on your marks.

Please remember that we make use of this progress section to give you your project mark. Your mark will suffer if you don't complete this section.

Semester 1 Week 1

Steven

  1. Week 1 work:
    • First meeting with Patrick and Derek
    • Introduced to Honours Project wiki page
      • Watched the 2010 project video
      • Watched the ABC documentary of the Somerton Man Case
      • Began brief review of 2009/2010 documentation
    • Patrick and I attended a Literature Search Training Session with Kay Leverett in the Barr Smith Library
      • Included a tutorial on how to use the engineering databases – very helpful.
  2. Goals for week 2:
    • Review previous years work in much greater detail
    • Clearly define the project scope in accordance with Derek’s suggestions in meeting
      • Generate a corresponding work breakdown structure
      • Summarise individual tasks and associated deliverables
    • Assist Patrick in preparing the Proposal Seminar ready for presenting in week 3.

Patrick

  1. Work Done this Week
    • Initial meeting with Steven and Derek
    • Had a brief read of 2009/10 Final Reports
    • Watched ABC Documentary of the Somerton Man case
    • Watched the 2010 group's YouTube video.
  2. Goals for Next Week
    • Detailed examination of Previous Years' work
    • Begin preparation of Proposal Seminar - to be presented in week 3.
    • Identify the Tasks and Paths of the project
      • Create Work Breakdown Structure
      • Divide up tasks between team members and assign deliverables to tasks

Semester 1 Week 2

Steven

  1. Week 2 work:
    • Attended meeting with Patrick and David Knight from the CS School
      • Discussed Web Crawler/Pattern Matcher implementation options and difficulties
      • Provided with additional resources for internet programming
    • Attended Occupational Health and Safety Lecture with Braden Phillips and Stephen Guest
      • Clarified requirements for project risk assessment and OH&S risk assessment for progress reports
    • Patrick and I planned the project approach
      • Established project work breakdown structure and proposed schedule
    • Analysed the Final Reports of the 2009 and 2010 projects, attention towards;
      • Understanding their procedures and results
      • Develop ideas for improvement or extension
    • Started work on Proposal Seminar
  2. Goals for week 3:
    • Prepare a gantt chart to represent proposed project schedule
    • Complete and present Proposal Seminar on Friday 18/03/11

Patrick

  1. Work Done this Week
    • Meeting with David Knight from the CS School
      • Discussed Web Crawler/Pattern Matcher implementation options and difficulties
    • Detailed analysis of 2009/10 Final Reports - Purpose to:
      • Summarise the main findings
      • Identify any areas missed in previous years.
    • Drew up Work Breakdown Structure and planned project approach
    • Started work on the Proposal Seminar.... Due to be presented next week
  2. Goals for Next Week
    • Complete and PRESENT Proposal Seminar
    • Begin work on verifying previous years' results


Semester 1 Week 3

Steven

  1. Week 3 work:
    • Work this week was devoted to the Proposal Seminar
    • Prepared gantt chart representing project schedule
    • Attended meeting with Patrick and Derek
      • Main focus on preparing Proposal Seminar
    • Completed and presented Proposal Seminar with Patrick – only one attendee, Derek
  2. Goals for week 4:
    • Establish software development environment on my laptop.
    • Begin review of 2009 and 2010 software development
    • Begin work on the Stage 1 Progress Report

Patrick

  1. Work Done this Week
    • Spent the majority of the week working on the Proposal Seminar
    • Had a meeting with Derek regarding Proposal Seminar
    • Downloaded and installed JDK and Java Eclipse on Laptop as a working/development environment for project work
    • Presented our Proposal Seminar! ... to one person, Derek. They don't know what they missed!
  2. Goals for Next Week
    • More work on Verifying previous results.
    • Begin code for text parser - the core of the pattern matcher
    • Start work on the Stage 1 Progress Report... does documentation ever end?

Semester 1 Week 4

Steven

  1. Week 4 work:
    • Began work on the Stage 1 Progress Report – due Friday Week 5
    • Established a SVN repository for the project
      • For use by Patrick and I as a code exchange point
    • Under Linux installed Java IDE for project use and establish a project working environment
    • Brief review of 2010 “web crawler” code
  2. Goals for week 5:
    • Requirements for another course will restrict my available time in Week 5
    • Complete Stage 1 Progress Report including integration onto wiki page
    • Continue with previous project software investigation if time permits

Patrick

  1. Work Done this Week
    • Began preparation of the Stage 1 Prog Report, due end of next week
    • Created simple text parser to work from.
      • Current ability:
        • removes punctuation
        • converts all letters to lowercase.
      • Successfully tested on a Rubaiyat text file!
  2. Goals for Next Week
    • Complete Stage 1 Progress Report and Submit/Put on wiki
    • Continue with text parser work

Semester 1 Week 5

Steven

  1. Week 5 work:
    • Limited availability this week due to requirements for another course
    • Worked heavily focused on the Stage 1 Progress Report
      • Integrated contributions from Patrick and myself and submitted to MyUni
  2. Goals for week 6:
    • Complete Stage 1 Progress Report
    • Integrate Stage 1 Progress Report onto the wiki page
    • Technical work:
      • Begin formulation of cipher scheme testing that will extend on results from 2009 and 2010
      • Begin exploration of additional cipher schemes that can be tested
        • Summarise methodology for testing validity

Patrick

  1. Work Done this Week
    • Lots of work on Stage 1 Progress Report
    • Submitted a semi complete copy to MyUni
    • Drew up design plans for pattern matcher and web crawler, and their integration
  2. Goals for Next Week
    • Finish Stage 1 Progress Report
    • Post up proper Stage 1 Progress Report onto the Wiki
    • Spotted a gap in Documentation - next one not due til week 12! Time for some proper work.
      • Look into statistically analysing some results gathered last year.
      • Continue work on text parser, start adding pattern matching features.

Semester 1 Week 6

Steven

  1. Week 6 work:
    • Completed and integrated the Stage 1 Progress Report to the project wiki page. See the link Patrick has provided below.
      • Learnt valuable wiki syntax and semantics
    • Formulated cipher scheme tests that will extend the results from 2009 and 2010
    • Identified possible additions to the cipher “cross-off list”
  2. Goals for week 7:
    • Meet with Derek and Patrick and plan availability and work for the mid-semester break
    • Begin implementation and testing of cipher schemes that will extend the results from 2009 and 2010
    • In collaboration with Patrick, begin implementation and testing additional cipher schemes for use in the “cross-off” list
    • Get a good result on my Engineering Management test.

Patrick

  1. Work Done this Week
    • Stage 1 Progress Report, Stage 1 Progress Report and more Stage 1 Progress Report. The completed report has been formatted for the Wiki site and is now available here.
    • Developed a plan to analyse the pattern matching results gathered in 2010. Unfortunately it relies on a working pattern matcher to gather more data, which leads to:
    • Further development of pattern matching software. An Initialise function has been added and pattern matching features are currently being designed.
  2. Goals for Next Week
    • Continue work on text parser/pattern matcher
    • Start testing Ciphers on the Cipher Cross-Off list
    • Set up a plan with Steven/Derek as to what we plan to achieve over the mid-semester break
    • Get a good result in my Financial Institutions Management test.

Semester 1 Week 7

Steven

  1. Week 7 work:
    • Attended a meeting with Derek and Patrick and summarised the work completed to date and intentions for the mid semester break
    • Patrick and I finalised use of the Cipher Cross-Off List as the means for presenting and centralising results of continued 2009 project cipher scheme testing. Specific page to be added to wiki by Patrick.
    • Investigated one cipher scheme from the “Cipher Cross-Off list”. Results provide more thorough proof of conclusions from 2009
  2. Goals for “break”:
    • Begin detailed research into web-crawler implementations – ensuring there is compatibility with Patrick’s pattern matcher software
    • Examine at least one more cipher on the Cipher Cross-Off list – results will be added to the page created by Patrick
    • Investigate possible additional university computer resources for use in developmental testing

Patrick

  1. Work Done this Week
    • Worked on text parser some more, developed a version that is capable of analysing the structural statistics of the Rubaiyat.
    • Used Text parser to gather statistics on the Rubaiyat. Results indicate that the code and the poems inside are NOT structurally related. See reports (to come) for further details.
    • Set up the wiki Cipher Cross-off List!
    • Had a meeting with Derek and Steven to go through our work done so far and to plan out work for the next period.
  2. Goals for Next Week
    • Test one cipher identified on the Cipher Cross-Off list
    • Further development of the Text Parser/Pattern Matcher
    • Look into creating a closed web environment in which our Web Crawler can be safely tested.
    • Enjoy my "holidays"

Semester 1 Mid-Semester Break

Steven

  1. Mid-semester break work:
    • Detailed examination of code used in cipher scheme testing for 2009 project
    • Completed testing of two more cipher schemes from the Cipher Cross-off List – results yet to be added to project wiki
    • Investigated potential web crawler implementations – research documentation needs further compilation
    • For the purposes of future testing contacted Braden Phillips and learnt how to apply for access to the university servers for additional computing power
  2. Goals for week 8:
    • Add results of cipher scheme testing to the project wiki
    • Test another cipher identified on the Cipher Cross-off list
    • Further investigation of web crawler implementations. In addition, update supporting documentation
    • Meet with Patrick and discuss compatibility of current web crawler options with regard to the pattern matcher software

Patrick

  1. Work Done this Fortnight
    • Tested and disproved a cipher on the Cipher Cross-off List - the Alphabet Reversal Cipher.
    • Started modifying text parsing code for html style web pages.
    • Researched web crawling requirements and created a broad web crawler/text parser concept document.
    • Looked into test environments for help in developing the web crawler and identified two possible options:
      • A closed environment in a local directory
      • Limit the crawling to only pages starting with the start URL. E.g. if the start URL is http://www.adelaide.edu.au, then it should only crawl websites starting with this address – no outside websites.
  2. Goals for Next Week
    • Test another cipher on the cross-off list.
    • Begin the development of a test environment for the web crawler.
    • More development on the text parser/pattern matcher.

Semester 1 Week 8

Steven

  1. Week 8 work:
    • Met with Patrick and concluded pattern matcher implementation can be adapted to any web crawler implementation
    • Tested another cipher on the Cipher Cross-off list – results yet to be added to wiki with Shift Cipher (Caesar Cipher) results and Auto-Key Cipher results.
    • Web crawler investigation postponed to week 9 pending limited availability.
  2. Goals for week 9:
    • Test another cipher on the cross-off list
    • Further web crawler investigation and supporting document development
    • Research HTML coding language for purpose of providing a personal web crawler testing environment
    • Complete a document summarising web crawler findings and development to week 10 – to be completed over week 9 and 10.


Patrick

  1. Work Done this Week
    • Disproved another cipher on the Cipher Cross-off List - the ADFGVX Cipher.
    • Began design and development of pattern-matching code and strategies.
    • Created a simple test environment for Web Crawl testing. It is a closed environment in a local directory containing multiple pages.
  2. Goals for Next Week
    • Test another cipher on the cross-off list.
    • Evaluate the pattern matching code developed by last year's group and produce a document summarising their efforts.
    • Continue the development of our own pattern-matching algorithms and code.

Semester 1 Week 9

Steven

  1. Week 9 work:
    • Tested and disproved another cipher scheme on the Cipher Cross-off List – the VIC Cipher
    • Began document summarising web crawler findings of 2010
    • Researched and implemented a small local HTML environment. Further expansion required for web crawler testing.
  2. Goals for week 10:
    • Begin Stage 2 Progress Report in accordance with planned schedule
    • Complete development of web crawler summarising document
    • Compile formal summarising document of the ciphers on Cipher Cross-off List tested thus far

Patrick

  1. Work Done this Week
    • Disproved another cipher on the Cipher Cross-off List - the Affine Cipher - Proof here. Resultant output of code has been uploaded here.
    • Examined and evaluated the 2010 pattern matching code - internal document produced.
    • Designed algorithm for Exact pattern matching function.
  2. Goals for Next Week
    • Test another cipher on the cross-off list.
    • Implement design of Exact pattern matching function.
    • Design 'wildcard' pattern matching function.
    • Begin work on the Stage 2 Progress Report as per Gantt Chart schedule.

Semester 1 Week 10

Steven

  1. Week 10 work:
    • In accordance with the planned schedule began compilation of the Stage 2 Progress Report
    • Compiled formal documentation of 2010 web crawler implementation
    • Worked on a document to summarise findings of cipher investigations for Cipher Cross-off List
  2. Goals for week 11:
    • Continue Stage 2 Progress Report as per the Gantt Chart
    • Prepare an updated Gantt Chart for use in the Stage 2 Progress Report
    • Get a copy of developmental software from Patrick – for use in document/presentation preparation
    • Test another cipher on the cross-off list
    • Continue development of the cipher investigation documentation
    • Update Cipher Cross-off List wiki page

Patrick

  1. Work Done this Week
    • Discounted another cipher on the Cipher Cross-off List - the Two-square Cipher.
    • Coded up and tested algorithm for Exact pattern matching function.
    • Began design of next pattern-matching algorithm, the 'wildcard' matcher.
    • Started documentation of Stage 2 Progress Report.
  2. Goals for Next Week
    • Identify and test another cipher for the cross-off list.
    • Complete design and begin implementation and test of wildcard pattern matching function.
    • Continue work on the Stage 2 Progress Report.

Semester 1 Week 11

Steven

  1. Week 11 work:
    • Investigated another cipher on the cipher cross-off list – the Bifid cipher
    • Continued Stage 2 Progress Report in accordance with planned schedule
    • Continued cipher investigation documentation
    • Prepared updated Gantt Chart for use in the Stage 2 Progress Report
  2. Goals for week 12:
    • Update cipher investigation documentation to include Bifid Cipher
    • Complete and submit Stage 2 Progress Report
    • Attend meeting with Derek and Patrick and summarise work completed to date and intentions for the mid-year break
    • Meet with Patrick and get a copy of developmental software

Patrick

  1. Work Done this Week
    • Disproved another cipher on the Cipher Cross-off List - the 4-rail Rail Fence Cipher.
    • Added and tested wildcard letter feature to Exact pattern matching function.
    • Stage 2 Progress Report reportedly progresses.
  2. Goals for Next Week
    • Identify and test another cipher for the cross-off list.
    • Design (and maybe begin implementation) of the next function for the pattern matching software - likely to be Generic Pattern Matcher - e.g. search for $%$% pattern returns ABAB, CDCD, GDGD etc.
    • Continue work on the Stage 2 Progress Report.
    • Complete and submit Stage 2 Progress Report.
    • Set up meeting with Steve and Derek to discuss progress and plans for the mid-year break.

Semester 1 Week 12

Steven

  1. Week 12 work:
    • Attended a meeting with Derek and Patrick, reviewed Stage 2 Progress Report and planned work for the Mid-year break.
    • Work heavily oriented towards Stage 2 Progress Report
      • Report should now be available on wiki
    • Completed analysis of another cipher - the Trifid Cipher
    • Obtained copy of developmental software from Patrick
      • Successfully installed!
    • Light beginning to flicker at the end of the tunnel...
  2. Goals for "break":
    • Test at least one cipher per week following exams
    • Adapt cipher analysis code for use with Cipher GUI
    • Implement Web Crawler prototype
    • Begin preparation of Poster and Final Seminar
    • Write up some ideas for the project video
    • Attend Somerton Man exhbition at the Bay Discovery Centre with Patrick

Patrick

  1. Work Done this Week
    • Met with Derek and Steven regarding Stage 2 Progress Report and work for the mid semester break.
    • Completed structural and statistical tests on the Rubaiyat.
    • Spent many an hour working on Stage 2 Progress Report
    • Handed up said report.
  2. Goals for Mid-Year Break
    • Identify and test multiple ciphers for the cross-off list.
    • Design, implement and test all of the remaining functions for the pattern matching software.
    • Work with Steven on the Web Crawler.
    • Implement Affine Cipher on Cipher GUI
    • Look at possibility of adding frequency plot graphs to the Cipher GUI.

Mid-Year Break

Steven

  1. Mid-year break work:
    • Attended “The Body on the Beach” exhibition at the Bay Discovery Center
    • Implemented following ciphers on Cipher GUI:
      • Auto-key Cipher
      • Bifid Cipher
      • Caesar Cipher
      • Hill Cipher
      • Trifid Cipher
      • Vigenere Cipher
    • Added functionality to the cipher GUI:
      • Modified layout to include areas for cipher information and letter frequency graphs
      • Load and storing of plaintext or ciphertext
      • Added information area about cipher techniques
    • Investigated several different prototype web crawlers
    • Investigated for the cipher cross-off list – the Rotary Phone Dial Cipher and the Nihilist Cipher
    • Acquired additional resources for the preparation of Poster(s)
  2. Goals for week 1:
    • Test another cipher for the cross-off list
    • Investigate and implement letter frequency graphs for integration into Cipher GUI
    • Meet with Patrick to review and reassess project schedule
    • If time permits - conduct further investigation of the web crawler prototypes to assess suitability
      • Findings to be documented accordingly

Patrick

  1. Work Done over Break
    • Added and ruled out several ciphers on the Cipher Cross-off List, including the Atbash Cipher, Baconian Cipher and Number-based ciphers
    • Completed all the Pattern Matching functions.
      • These include:
        • ExactMatch - to detect exact String matches, e.g. a search for "ABAB" is successful in "Kebab ahoy!"
        • ExactMatchInitial - the same as ExactMatch but only uses initial letters in detections
        • PatternMatch - to detect patterns in Strings, e.g. a search for "ABAB" is successful in "Crisis"
        • PatternMatchInitial - identical to PatternMatch but only uses initial letters in detecting
      • These functions have all been successfully tested
      • I have written a short report detailing the workings of the code as well as it's strengths and weaknesses
    • Investigated some methods for including bar charts (frequency plots) in the Cipher GUI
    • Set up the project's svn repository to encourage collaborative work, and added the pattern matcher software to it
    • Visited the Somerton Man Photography and Paint exhibition titled "The Body on the Beach" at the Bay Discovery Center. Very thought-provoking and intriguing.
  2. Goals for Next Week
    • Identify and test more ciphers for the cross-off list
    • Start looking at integrating pattern matching code into a candidate web crawler
    • Bug Steven to add the Cipher GUI to the svn repository so I can work on it and implement some ciphers
    • Review the goals and timelines of the project to ensure we are still on track

Semester 2 Week 1

Steven

  1. Week 1 work:
    • Investigated web crawler prototypes with testing completed in closed local environment
    • Investigated another cipher for the cipher cross-off list – The Trithemius Cipher
    • Implemented Trithemius Cipher on Cipher GUI
    • Implemented Java based letter frequency graphs for Cipher GUI
    • Met with Patrick and reaffirmed project development and schedule
  2. Goals for week 2:
    • Test another cipher for the cross-off list
    • Prepare and submit the Project Exhibition requirements (due Friday August 5)
    • Prepare and submit brochure description and image for Project Exhibtion (due Friday August 5)
    • Add letter frequency graph functionality to Cipher GUI
    • Add Cipher GUI code to project SVN repository
    • Meet with Patrick and outline how additional user functionality will be added to CipherGUI

Patrick

  1. Work Done this Week
    • Added and ruled out several ciphers on the Cipher Cross-off List, including the Pigpen Cipher and Homophonic Substitution Cipher
    • Discussed details of Web Crawler implementations with Steve and how the pattern matching code is likely to integrate.
    • Evaluated current project progress regarding key arms, including the Web Crawler/Pattern Matcher side and the Cryptanalysis sides. Both seem to be progressing well and on schedule.
  2. Goals for Next Week
    • Identify and test more ciphers for the cross-off list
    • Begin integrating pattern matching code into a candidate web crawler
    • Implement the Affine Cipher in the Cipher GUI
    • Prepare Exhibition information - due Friday Week 2 - regarding the project exhibition in week 12. This includes:
      • A brief description about the project and a small picture both to go in the Exhibition Guide brochure
      • Submit requirements for our stall at the exhibition

Semester 2 Week 2

Steven

  1. Week 2 work:
    • Analysed and added the Templar Cipher to the Cipher Cross-off List
    • Management and documentation:
      • Compiled and submitted Project Exhibition requirements
      • Prepared and submitted Project Exhibition brochure description and image
    • Work for Cipher GUI:
      • Added letter frequency graph functionality to Cipher GUI
      • Added all Cipher GUI code to project SVN repository (following connection issues between my laptop and Electrical Engineering Department Server)
      • Met with Patrick and established how additional user functionality – entering keywords etc. will be added to Cipher GUI
  2. Goals for week 3:
    • Test another cipher for the cross-off list
    • Implement cipher on cipherGUI if applicable
    • Identify and test candidate web crawlers suitable for use with Pattern Matching code
    • Modify Cipher GUI layout to enable cipher interactivity capabilities

Patrick

  1. Work Done this Week
    • Investigated another cipher for the Cipher Cross-off List - the Book Cipher.
    • Cipher GUI work:
      • Implemented the Alphabet Reversal Cipher (aka Atbash Cipher) on the Cipher GUI
      • Began implementing the Affine Cipher on the Cipher GUI - more work required (see Goals for next Week)
      • Discussed with Steven the future development of the Cipher GUI - namely how we are going to add interactive abilities for the user for ciphers that involve the choice of a keyword/key number.
    • Web Crawler/Pattern Matcher work:
      • Not much progress this week as still waiting on a candidate web crawler to begin working on.
    • Management and Documentation work:
      • Aided Steven in designing the brief project description and picture to be put in the Exhibition day Brochure
      • Helped Steven with identifying our requirements for the Project Exhibition.
  2. Goals for Next Week
    • Investigate another cipher for the Cipher Cross-off List
    • Add interactivity to the Affine Cipher in the Cipher GUI allowing users to pick which shifts and multipliers are used in encrypting/decrypting
    • Identify and investigate a candidate web crawler within which we will implement the Pattern Matching code

Semester 2 Week 3

Steven

  1. Week 3 work:
    • Investigated another cipher for the cipher cross-off list – the Beaufort Cipher
    • Work for Cipher GUI:
      • Implemented Beaufort Cipher on Cipher GUI
      • Worked with Patrick to develop an interactive panel for user to control cipher characteristics where applicable
    • Work for Web Crawler/Pattern Matcher:
      • Tested use of pattern matcher with regard to use in three candidate web crawlers
      • Met with Patrick and passed on the three alternative web crawlers for him to perform independent testing and assess suitability
  2. Goals for week 4:
    • Design and implement a prototype graphical user interface for the web crawler
    • Investigate implementation of pattern matcher code in three candidate web crawlers
    • If time permits, begin converting ciphers implemented on Cipher GUI to include interactive cipher capabilities

Patrick

  1. Work Done this Week
    • Investigated another cipher for the Cipher Cross-off List - the Null Cipher.
    • Cipher GUI work:
      • Developed the blueprint for adding user functionality to the Cipher GUI (though no blueprint for using keywords for ciphers yet).
        • Namely by way of designating a section of the Cipher GUI that each cipher can form as it chooses.
      • Used this to finish off the Affine Cipher implementation. Fully functional and tested!
    • Web Crawler/Pattern Matcher work:
      • Downloaded the three Web Crawlers Steve recommended.
  2. Goals for Next Week
    • Investigate another cipher for the Cipher Cross-off List
    • Begin upgrading some of the currently implemented ciphers on the CipherGUI to match the newest interactive nature
    • Investigate the web crawlers downloaded this week.

Semester 2 Week 4

Steven

  1. Week 4 work:
    • Work for Web Crawler/Pattern Matcher:
      • Designed and implemented a prototype web crawler GUI
      • Investigated implementation of pattern matcher code in three candidate web crawlers
    • Cipher GUI:
      • No time permitting for conversion of existing Cipher GUI ciphers to include interactive key functionality
  2. Goals for week 5:
    • Test another cipher for the cross-off list
    • Implement tested cipher on Cipher GUI
    • Upgrade some of the ciphers currently implemented on the Cipher GUI to include interactive key functionality
    • Consult with Patrick to further investigate identified web crawlers for suitability and adaptation with pattern matching code

Patrick

  1. Work Done this Week
    • Investigated another cipher for the Cipher Cross-off List - the Four-square Cipher.
    • Cipher GUI work:
      • Added unique panels to both the Alphabet Reversal/Atbash and Auto-key Cipher implementations on the GUI - enabling user interactivity for choosing keys.
    • Web Crawler/Pattern Matcher work:
      • Attempted to investigate the 3 web crawlers Steve advised me to have a look at. Failed on 3 accounts, mainly due to install issues.
  2. Goals for Next Week
    • Investigate another cipher for the Cipher Cross-off List
    • Upgrade some more of the currently implemented ciphers on the CipherGUI to add interactive key choices
    • Get help from Steve in investigating these web crawlers!
    • In consultation with Steve, begin identifying:
      • Which web crawler(s) we'll base ours around
      • Where we will integrate pattern matching code

Semester 2 Week 5

Steven

  1. Week 5 work:
    • Analysed another cipher for the cipher cross-off list – the Gronsfeld Cipher
    • Work for web crawler/pattern matcher:
      • Met with Patrick and discussed web crawler GUI. Changes required – see goals for week 6
      • Attempted to run web crawler candidates from university computers (through proxy) – unable to get one of three running. Need to consult with Derek regarding a mutual account
    • Work for Cipher GUI:
      • Implemented Gronsfeld Cipher on CipherGUI
      • Modified several Cipher GUI ciphers to include interactive key functionality
  2. Goals for week 6:
    • Test another cipher for the cross-off list
    • Implement cipher on cipher GUI
    • Continue upgrading existing ciphers within the Cipher GUI to include interactive key functionality
    • Modify web crawler GUI to include area for additional search patterns
    • Meet with Derek to discuss web crawler implementation and account access for testing at the university

Patrick

  1. Work Done this Week
    • Investigated another cipher for the Cipher Cross-off List - the Dvorak Cipher.
    • Cipher GUI work:
      • Implemented Dvorak Cipher on the Cipher GUI.
    • Web Crawler/Pattern Matcher work:
      • Continued to work on investigating identified web crawlers. Got one working well.. crawler4j looks promising. Was able to crawl web pages successfully.
      • Discussed with Steve issues involving the implementation of a web crawler (and GUI) and its integration with pattern matching code.
  2. Goals for Next Week
    • Investigate another cipher for the Cipher Cross-off List
    • Add investigated cipher to the Cipher GUI
    • Upgrade some more of the currently implemented ciphers on the CipherGUI
    • Continue to investigate web crawler implementations and functionality
    • Meeting with Steve and Derek to discuss future directions of the project.

Semester 2 Week 6

Steven

  1. Week 6 work:
    • Investigated another cipher for the cipher cross-off list – the Porta Cipher
    • Work for Cipher GUI:
      • Implemented Porta Cipher on Cipher GUI
      • Unable to make good progress with upgrading existing ciphers to include interactive key functionality – focus next week
    • Work for web crawler/pattern matcher:
      • Modified web crawler GUI to include area for additional search patterns
    • Management and documentation:
      • Attended a meeting with Derek and Patrick and summarised the work completed to date. Changes to be made to cipher GUI – see goals for week 7. Next meeting scheduled for Week 9.
  2. Goals for week 7:
    • Add cipher names to titles of letter frequency graphs in Cipher GUI in accordance with recommendations from meeting
    • Continue upgrading existing ciphers within the Cipher GUI to include interactive key functionality
    • Update information panels of remaining ciphers implemented in Cipher GUI
    • Commence final report (for submission 21/10/11)

Patrick

  1. Work Done this Week
    • Investigated another cipher for the Cipher Cross-off List - the Keyword Cipher.
    • Re-organised the Cipher Cross-off List - finished off linking and put all ciphers alphabetically
    • Cipher GUI work:
      • Added Keyword Cipher to the GUI - including keyword functionality
      • Cleaned up some of the other implemented ciphers - namely Atbash, Affine, Autokey, Dvorak
      • Edited Barchart code to enable the Frequency Plots to scale with the data.
    • Web Crawler/Pattern Matcher work:
      • Further investigated crawler4j. Seems to have most the functionality required.
    • Project Management:
      • Had a meeting with Derek and Steve. General feeling that we're satisfactorily on track.
  2. Goals for Next Week
    • Investigate another cipher for the Cipher Cross-off List
    • Implement a cipher on the Cipher GUI
    • Further investigate the web crawler
    • Identify ways to interface between the pattern matching code and the web crawler output.

Semester 2 Week 7

Steven

  1. Week 7 work:
    • Work for Cipher GUI:
      • Added cipher names to the titles of letter frequency graphs in accordance with recommendations from Derek
      • Updated information panels for the Beaufort Cipher, Porta Cipher, Shift Cipher, Trithemius Cipher and Vignere Cipher
      • Added image magnification functionality to the Porta and Trithemius Cipher information panels
      • Added interactive key functionality for the Bifid and Trifid Ciphers – information areas still need to be updated
      • All changes above have been committed to the project repository
    • Management and documentation:
      • Commenced final report (to be submitted 21/10/11)
  2. Goals for week 8:
    • Implement a cipher on Cipher GUI
    • Implement interactive key functionality for remaining ciphers on the Cipher GUI
    • Continue work on web crawler
    • Continue final report
    • Commence work on final seminar presentation to be held 30/09/11

Patrick

  1. Work Done this Week
    • At Derek's suggestion, had a second look at one of the ciphers investigated in 2009 - the Playfair Cipher. No conclusions yet.
    • Cipher GUI work:
      • Added the Playfair Cipher to the GUI - was quite complex to add including the ability to specify keywords to form the matrix.
      • Fixed a few small annoying bugs in the cipher GUI.
    • Web Crawler/Pattern Matcher work:
      • Looked into what pages the crawler (crawler4j) can access
      • Examined how the crawler (crawler4j) returns page content - so that we can tap into it to pattern match
  2. Goals for Next Week
    • Further investigation of ciphers for the Cipher Cross-off List
    • Implement a cipher on the Cipher GUI
    • Continue working with the web crawler
      • Hopefully develop a prototype implementation that can pattern match web page content

Semester 2 Week 8

Steven

  1. Week 8 work:
    • Work for Cipher GUI:
      • Implemented the Double Playfair Cipher including interactive key functionality in accordance with request from Derek after previous meeting
      • Added interactive key functionality for Hill Cipher – all ciphers in Cipher GUI now have interactive key functionality!
    • Work for web crawler/pattern matcher:
      • Redesigned web crawler GUI to include:
        • Tab layout to separate seed, pattern and saving controls
        • Ability to add/edit/remove multiple search seeds
        • Capacity to add/edit/remove multiple search patterns
        • Control over number of pages to search
        • Control over search depth
      • Began integrating the web crawler/pattern matcher into the GUI frontend
      • Examined resource highlighted by Matthew – search engine company blekko allowing public submission of regular expressions to analyse
    • Management and documentation:
      • Continued final report (to be submitted 21/10/11)
      • Decided with Patrick to postpone commencement of final seminar until next week to better accommodate advice from the scheduled meeting
  2. Goals for week 9:
    • Meet with Derek and Patrick to discuss progress and final seminar
    • Commence work on presentation for final seminar to be held 30/09/11
    • Upload investigated ciphers to cross-off list (not uploaded earlier as I had no access to project page)
    • Create demonstration web crawler/pattern matcher GUI
    • Formulate some regular expressions with Patrick that can be submitted to the blekko analysis tool
    • Update information panels in Cipher GUI

Patrick

  1. Work Done this Week
    • Examined the Playfair Cipher using the code and the Cipher GUI updated last week. Concluded we can rule it out. Click here for reasoning.
    • Cipher GUI work:
      • Fixed a bug in Playfair Cipher relating to forming the Playfair Square
      • Changed the default plaintext display to lowercase and ciphertext display to uppercase as per cryptological norms.
    • Web Crawler/Pattern Matcher work:
      • Completely re-designed and re-coded pattern matching code for the following reasons:
        • Make the setup neater, easier to understand and more flexible to change/add functionality.
        • New understanding on how the web crawler provides data enabled a more direct design approach.
      • Created a demo pattern matching web crawler that successfully detected patterns in a live Web-Crawling test!
      • Had a look at an article, pointed out by Matthew, regarding a similar project by the search engine company blekko that allows users to submit a regex to run a search on.
  2. Goals for Next Week
    • Further investigation of ciphers for the Cipher Cross-off List
    • Implement a cipher on the Cipher GUI
    • Work with Steve in integrating the web crawler/pattern matcher into a GUI
    • Upgrade the pattern matching code further
    • Come up with some regex's we can submit to the blekko crawler to see if we can get them to test for us
    • Meet with Steve and Derek regarding progress to date and plans for the Final Seminar
    • Begin work on said Final Seminar.

Semester 2 Week 9

Steven

  1. Week 9 work:
    • Work for Cipher GUI:
      • Modification of Cipher GUI postponed to ensure there are no conflicting versions with the updates Patrick plans to make
    • Work for web crawler/pattern matcher:
      • Met with Patrick and reviewed the web crawler GUI
      • Continued integrating the web crawler and pattern matching software into the GUI frontend
      • Reconfigured web crawler and pattern matching programs for improved user operation and feedback
      • Added the ability to select exact and initial pattern definitions into the web crawler GUI
      • Reviewed the blekko analysis tool highlighted by Matthew and concluded a submission would not be made. The major factors were the inability to control the web crawl and uncertainty as to whether our submission would be used.
      • Formulated regular expressions initially for use with blekko analysis tool. Now will be used with the additional pattern matching functionality that Patrick is developing.
    • Management and documentation:
      • Attended a meeting with Derek and Patrick and summarised the progress to date. Plans for the final seminar were also discussed.
      • Commenced work on presentation for final seminar (30/09/11)
  2. Goals for "break":
    • Week 1 (final seminar 30/09/11)
      • Continue compilation of final seminar presentation
      • Present the final seminar on Friday 30/09/11
      • Complete integration of web crawler, pattern matcher and graphical user interface
      • Update web crawler graphical user interface to include regular expression functionality
      • Any further software modifications that are required for the demonstration component of the final seminar
    • Week 2 (final seminar complete)
      • Meet with Patrick and review all work required for project closeout
      • Begin poster(s) for the final exhibition (28/10/11)
      • Plan project video – acquire all additional resources
      • Work with Patrick to establish how our software can be deployed on the project wiki
      • Continue final report (submission 21/10/11)

Patrick

  1. Work Done this Week
    • Cipher Cross-off List
    • Cipher GUI work:
      • Added Substitution Cipher that lets the user pick their own Cipher Alphabet.
      • Improved Keyword Cipher by added a JTable that displays the current Cipher Alphabet according to the user-entered keyword.
      • Improved Playfair Cipher by addinga Table that shows the Playfair square. The table updates when the Keyword is changed by the user.
      • Added the unique center panel for the Caesar Cipher.
    • Web Crawler/Pattern Matcher work:
      • Discussed the Crawler GUI setup with Steve who is working on the integration into the GUI
      • Investigated Regular Expressions (regexs) for the purpose of using them for a crawl on blekko's web data as identified last week
        • Identified Regexs that could successfully match to Initial Letters and character patterns
        • Decided that blekko wouldn't be much help since they aren't providing the option to run a web crawl, they are just offering to run two crawls a day themselves.
        • However these regexs have given me an idea and the production of an additional Pattern Matching feature is under way! A Regex option - crawl the web for a regex of your choice.
    • Project Management work:
      • Had a meeting with Steve and Derek. Addressed progress on work to date, and discussed plans for the final seminar coming up.
      • Began planning on the content of the Final Seminar.
  2. Goals for Next Fortnight
    • Week 1
      • Work on the Final Seminar
      • More work on the Final Seminar
      • Even More work on the Final Seminar! Present the Seminar on Friday.
      • Look into recording our Seminar so Derek (who is overseas) can watch it.
      • Implement a Regular Expression Pattern Matching option in the web crawler.
      • Do whatever other final touches are needed for the Web Crawler/Pattern Matcher and Cipher GUI for the Final Seminar's Demonstration section.
    • Week 2 (Seminar over)
      • Meet with Steven to identify work still needed to finish off the project - review progress on the Gantt Chart
      • Begin work on the Poster(s) for the Project Exhibition.
      • Begin planning our project Video
      • Look into how we can host our software on the internet

Semester 2 Mid-Semester Break

Steven

  1. Mid-semester break work:
    • Week 1:
      • Significant work conducted on presentation for final seminar – rehearsals conducted over three days
      • Presented final seminar (30/09/11)
      • Modified web crawler and pattern matcher for demonstration with final seminar
      • Integrated the web crawler, pattern matcher and GUI frontend into a single piece of software
      • Added regular expression functionality to web crawler graphical user interface
      • Downloaded videos for possible use in project video
    • Week 2:
      • Project video:
        • Analysed 2009 and 2010 project videos.
        • Commenced planning of project video.
        • Acquired more potential videos for use in the project video.
      • Project poster:
        • Examined 2009 and 2010 poster designs
        • Commenced formulation of a layout for the project poster.
        • Researched possible content for additional poster(s).
      • Reviewed and updated the cipher cross-off list.
      • Continued final report (to be submitted 21/10/11)
  2. Goals for week 10:
    • Attend project exhibition briefing with Braden Phillips (11/10/11)
    • Meet with Patrick to discuss requirements for project closeout
    • Add webpage prefix filtering capability to the web crawler
    • Continue poster(s) for the final exhibition
    • Continue final report
    • Complete conversion and upload of final seminar video onto YouTube.

Patrick

  1. Work Done this Fortnight
    • First Week
      • Plenty of work on Final Seminar; for a "Holidays week" in at uni each day but Monday!
      • Gave Final Seminar; was well received
      • Downloaded software to enable recording of Final Seminar for Derek who is overseas.
      • Implemented Regular Expression Pattern Matching into the Web Crawler!
      • Fixed+Modified other issues with Web Crawler/Pattern Matcher in preparation for Final Seminar demonstration.
    • Second Week
      • Poster:
        • Did background research for our project poster, including examining multiple previous year examples
        • Identified some possible setups for our main project poster
        • Identified possible content for a second poster
      • Video:
        • Did background research for the planned video including analysing previous years' videos
        • Began planning some content ideas for our video
      • Hosting Software:
        • Successfully packaged the current Pattern Crawler into a single executable jar file
        • A lot of work integrating the CipherGUI into a single jar file, including its images. Successfully integrated all but two ciphers so far.
        • Looked into hosting software on the wiki page. Due to limited acceptable Upload file types, they will most likely have to be uploaded as jar files in a zipped package.
  2. Goals for Next Week
    • Finish off enabling the CipherGUI to be compiled into a single executable jar file
    • Help Steven with numerous bug fixes and required functionality implementations in the Pattern Crawler...
    • Design pattern matching code to enable a search for multiple patterns on the same page - e.g. to return a result both patterns must be present.
    • Finish compiling/packaging/editing the Final Seminar video and host it on YouTube so others (inc. Derek and Matthew) can view it.
    • Attend lecture on Exhibition to be held Tuesday
    • Continue work on Posters for Exhibition
    • Continue work on project Video
    • Begin work on Final Report; due end Week 11.

Semester 2 Week 10

Steven

  1. Week 10 work:
    • Attended project exhibition briefing with Braden Phillips
      • Clarified timetable of day (28/10/11) and requirements for poster(s)
    • Met with Patrick and discussed project posters, project exhibition, project video and deployment of software on the project wiki
      • Decided on two posters and their layouts for project exhibition.
    • Modified web crawler and pattern matcher to include webpage filtering.
      • A webpage prefix can now be specified in interface to control crawling behaviour.
    • Continued final report (to be submitted 21/10/11)
    • Formatted video of the final seminar into three parts and converted each to a type suitable for YouTube
  2. Goals for week 11:
    • Meet with Derek and Patrick to discuss progress, final seminar video, final report and the final exhibition
    • Attend School of Electrical and Electronic Engineering final year photograph (27/10/11)
    • Complete and submit Final Report
    • Continue posters for the final exhibition (28/10/11)
    • Continue project video

Patrick

  1. Work Done this Week
    • Added in the Gronsfeld Cipher's unique description panel in the CipherGUI.
    • Fixed pattern matching bugs in pattern crawler code
    • Modified the layout of some pattern matcher code to make it uniform with other implementations
    • Successfully set up CipherGUI for executable jar file format
    • Met with Steven to discuss poster/video layouts/content
    • Attended Exhibition lecture; got a pep talk from Braden about what is expected
    • Worked with Steve on Final Seminar video and uploaded them to YouTube
    • Final Report commenced
  2. Goals for Next Week
    • Meet with Derek and Steven to discuss project progress to date and closeout
    • Finish and submit Final Report
    • Run pattern matching tests on code segments using complete web crawler.
    • Aid Steve with work on posters/video

Semester 2 Week 11

Steven

  1. Week 11 work:
    • Week committed to the final report. Submitted to myUni Friday night.
    • Reformatted and compiled the final report onto wiki page.
    • Attended a meeting with Derek and Patrick and summarised current project progress, project closeout, the final report and the final seminar including the project video and poster.
    • Attended School of Electrical and Electronic Engineering final year photograph.
    • Unable to make any further progress with posters or video this week – focus next week.
  2. Goals for week 12:
    • Complete project poster for the project exhibition – a “theme” is yet to be decided.
    • Complete “general interest” poster. Provides information about three other famous ciphers – Zodiac, Beale and Kryptos.
    • Complete project video for use in the project exhibition
    • Attend project exhibition on Friday 28/10/11
    • Assist Patrick to add executable jar files of the CipherGUI and the web crawler/pattern matcher to the project wiki webpage.
    • Provide Derek with a disc of all project software and electronic files.

Patrick

  1. Work Done this Week
    • Many an hour on Final Report, submitted Friday night.
    • Many more hours spent reformatting Final Report and compiling onto the wiki page
    • Worked on cleanup and modification of code for both CipherGUI and Web Crawler for Final Report purposes
    • Ran pattern searches on Somerton Code portions using Web Crawler
    • Met with Steven and Derek. Discussed project outcomes, final work to complete, Final Report and video/poster ideas
  2. Goals for Next Week
    • Finish off CipherGUI - namely Trifid/Bifid descriptions and interactive tables
    • Attempt to provide proxy options for Web Crawler to enable crawling on the University network
    • Aid Steve with Poster for Exhibition
    • Aid Steve with Video production
    • Attend Exhibition on Friday 28/10/2011
    • Upload CipherGUI and Web Crawler executable jar files onto project wiki
    • Handover all project products and work to Derek

Semester 2 Week 12

Steven

  1. Week 12 work:
    • Web crawler/pattern matcher:
      • Added proxy interface fields to the web crawler graphical user interface.
      • Assisted Patrick in the investigation of the proxy settings for crawling on the uni network.
      • Created two alternative executable jar files of Crawler
        • One that is a single jar file containing all required libraries. Too large to add to the project webpage.
        • A second split version that segments external libraries from the operational functionality. Will be added to the wiki for public access.
    • CipherGUI:
      • Tested feasibility of compiling CipherGUI into single executable jar file.
      • Created single executable CipherGUI jar file as insurance. Not required - Patrick's version to be uploaded to wiki.
    • Project Exhibition:
      • Worked with Patrick to compile a project poster with the theme of a 40’s era investigation wall.
      • Worked with Patrick to complete second “general interest” poster containing three other famous ciphers – Zodiac, Beale and Kryptos. Uses the same theme as the project poster.
      • Attended Project Exhibition on Friday 28/10/11. Well received with a lot of interest. Unfortunately no prizes.
    • Completed project video. Will be added to the final seminar videos on YouTube and a link from the wiki provided.
    • Management and documentation:
      • Attended a final meeting with Derek and Patrick to conclude project closeout and handover a CD containing all project software and electronic files for the year.
      • Exams next week. Thank you to both Derek and Patrick for all your help throughout the year and a great project.

Patrick

  1. Work Done this Week
    • CipherGUI work:
      • Added descriptions and interactive tables to both Bifid and Trifid ciphers
      • Re-coloured all cipher panels for consistency and re-organised central class naming setup of CipherGUI
      • Wrote up README.txt file instructing use of CipherGUI
      • Created final finished product executable jar file of GUI
      • Uploaded CipherGUI onto project wiki-site available here: Cipher GUI
    • Web Crawler work:
      • Investigated proxy settings for crawling on the uni network
      • Successfully identified correct settings (for uni) and designed a way to allow the user to input these into the GUI
      • Wrote up README.txt files - instructions for others to use our crawler.
      • Created final product executable jar file of Crawler
      • Uploaded Crawler jar file onto project wiki-page, see Web Crawler 2011
    • Exhibition
      • Aided Steven with Poster designs and contents for the main project poster
      • Wrote up Beale story and Zodiac Killer story for the Additional poster
      • Got Additional Poster printed at Officeworks, nice and shiny on glossy paper!
      • Attended exhibition and showed off products to plenty of interest
    • Other:
      • Uploaded project video to YouTube
      • Slept not much
      • Had final meeting with Derek and Steve and handed over CD containing all my work throughout the year.
  2. Goals for Next Week
    • Start studying for exams!
    • Over and out. Thanks for the brilliant experience this project has been.

See also

Back

Retrieved from "https://www1.eleceng.adelaide.edu.au/personal/dabbott/wiki/index.php?title=Cipher_cracking_2011_weekly_progress&oldid=7927"