Talk for me application 2024
Contents
Supervisors
Honours students
General project description
The Talk For Me app provides a fully customisable yet easy to use grid to point to communicate for people who can't talk, and supports text-to-speech output and image-recognition suggested text. This project is about extending the capability further, from words to sentences, possibly involving the use of ChatGPT. You will require a Mac computer since this is an iOS app, and familiarity with Swift and SwiftUI is essential.
Specific tasks
The tasks identified by the group to complete for this project focus around improving three areas, the UI, the functionality of the App, and the performance of the app through LLM optimisation.
UI
- Update the UI to fit with similar apps on the market
Functionality of the App
- Receive Location/Time information to make informed suggestions
- Suggest Menu Items based off the users location
- Suggest previously used sentences given a selection of words
- Generate 1+ messages from the LLM
- Sort the keywords presented to the users based off the acquired Location/Time information gained above.
Large Language Models
- Run and compare the time take to respond over a number of datasets reflecting 1-3 word combinations
- Compare the number of outliers in each performance
- Compare the models "reliability" as in how well it follows the provided instructions to respond with a response that contains the expected words (and a sentence that can be used for TTS).
Deliverables
Semester 1
- Project Proposal File:TalkForMe-ProjectPlan.pdf
- Seminar [1] password: 3LC!+USl
- Progress Report: File:TalkForMe-ProgressReport.pdf
Semester 2
- Ingenuity Poster File:Talk For Me Ingenuity Poster.pdf
- Ingenuity Video
- Final Report File:TalkForMe-FinalReport.pdf
Weekly progress and questions
This is where you record your progress and ask questions. Make sure you update this every week.
Approach and methodology
The approach involved a number of methods listed in our Progress and Final Reports. Due to our NDA, strict methods are not able to be provided, but can be summarised as accessing the existing code, and modifying/adding to it to gain the desired functions/changes.
Relationship to possible career path
The the coding and project management skills you pick up in the project will serve you for a wide range of careers.
References and useful resources
If you find any useful external links, list them here:
