Difference between revisions of "Talk for me application 2024"
(→Approach and methodology) |
(→Deliverables) |
||
| Line 36: | Line 36: | ||
===Semester 1=== | ===Semester 1=== | ||
| − | * | + | * Project Proposal [[File:TalkForMe-ProjectPlan.pdf]] |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
===Semester 2=== | ===Semester 2=== | ||
Revision as of 13:06, 5 November 2024
Contents
Supervisors
Honours students
General project description
The Talk For Me app provides a fully customisable yet easy to use grid to point to communicate for people who can't talk, and supports text-to-speech output and image-recognition suggested text. This project is about extending the capability further, from words to sentences, possibly involving the use of ChatGPT. You will require a Mac computer since this is an iOS app, and familiarity with Swift and SwiftUI is essential.
Specific tasks
The tasks identified by the group to complete for this project focus around improving three areas, the UI, the functionality of the App, and the performance of the app through LLM optimisation.
UI
- Update the UI to fit with similar apps on the market
Functionality of the App
- Receive Location/Time information to make informed suggestions
- Suggest Menu Items based off the users location
- Suggest previously used sentences given a selection of words
- Generate 1+ messages from the LLM
- Sort the keywords presented to the users based off the acquired Location/Time information gained above.
Large Language Models
- Run and compare the time take to respond over a number of datasets reflecting 1-3 word combinations
- Compare the number of outliers in each performance
- Compare the models "reliability" as in how well it follows the provided instructions to respond with a response that contains the expected words (and a sentence that can be used for TTS).
Deliverables
Semester 1
- Project Proposal File:TalkForMe-ProjectPlan.pdf
Semester 2
- Final seminar (Week 10)
- Final report (Week 11) - only one report needed in wiki format
- Poster (Week 12) - one poster only needed
- Project exhibition 'expo' (Week 12)
- CD or stick containing your whole project directories (Week 13)
- YouTube video (Week 13) - add the URL to this wiki
Weekly progress and questions
This is where you record your progress and ask questions. Make sure you update this every week.
Approach and methodology
The approach involved a number of methods listed in our Progress and Final Reports. Due to our NDA, strict methods are not able to be provided, but can be summarised as accessing the existing code, and modifying/adding to it to gain the desired functions/changes.
Possible extension
TBA
Expectations
- To get good marks we expect you to show a logical approach to coding.
- In your conclusion, you need to come up with a short list of likely possibilities and a list of things you can definitely eliminate.
- We expect all the written work to be place on this wiki. No paper reports are to be handed up.
- It is expected that you fill out a short progress report on the wiki each week, every Friday evening, to briefly state what you did that week and what the goals are for the following week.
- It is important to regularly see your main supervisors. Don't let more than 2 week go by without them seeing your face briefly.
- You should be making at least one formal progress meeting with supervisors per month. It does not strictly have to be exactly a month, but roughly each month you should be in a position to show some progress and have some problems and difficulties to discuss.
- The onus is on you to drive the meetings, make the appointments and set them up.
- You are expected to make a YouTube presentation of your whole project.
Relationship to possible career path
The the coding and project management skills you pick up in the project will serve you for a wide range of careers.
References and useful resources
If you find any useful external links, list them here:
