Deep reinforcement learning is a branch of machine learning that focuses on learning behaviours and has recently become a powerful framework for solving high-dimensional problem spaces. The Arcade Learning Environment (1995) was one of the first breakthroughs of Deep Reinforcement Learning that used a Deep Q-Network to learn how to play a series of different Atari games effectively. This project built upon this foundation by proposing a model that utilises more modern Reinforcement Learning algorithms, such as Double Deep Q-Learning and REINFORCE, while allowing user customisation of parameters.

The project software was organised to allow for advanced customisation of all parameters, which allowed the user to effectively examine the effects of altering different values on the agent's performance. Each playthrough stored the specified parameters in JSON format, allowing the user to easily replay previous iterations and alter parameters. The project was written in Python and utilised complex PyTorch operations to achieve efficient results

Overall, the project was a success and you can run the agent for yourself by visiting my GitHub account.

Managing contacts has always been a challenge, especially for smaller companies who can't afford the large licensing fees required to run state-of-the-art contact management software. Over the summer of 2022 I worked with a small company, IMS Publications, to completely overhaul their current system and instead host a bespoke website that handles every transaction between themselves and their clients.

The main challenge with installing a completely new system is data migration. For the past 26 years, an old system (Tracker v2.0) has been used, which is so ancient that there is no documentation of it on the internet. Hence, obtaining any access to the database via inspecting the database files were completely out of the question; all files were written in a format that is 25 years deprecated. To solve this issue, I developed an advanced autoclicker software that visually analysed every record on the database and automatically generated a new record on the new InTouch system. Over 65,000 records were successfully transferred onto the new system.

For my 4th year group project, we decided to tackled the issue with new students being overwhelmed during their initial experience at the Universities, more specifically the University of Leeds which this application was designed around. This project presented an innovative Chatbot: combining IBM technologies, Artificial Intelligence and Augmented Reality into one Unity mobile application. The application will instantly answer the University-related requests of new students, guide them to their lecture halls in augmented reality, recommend societies most suited to them and more. These features are presented through an interactive virtual avatar, also available in augmented reality. This application also offers a plethora of accessibility features, from language translation, speech-to-text and colour-blind modes.

My main contributions to the project were handling all unity application details, user interactions and integration of different technologies. The user was able to interface with the chatbot in several different ways:

1. Augmented reality- an avatar was superimposed onto the camera view, where the user could ask questions via keyboard input or speed-to-text, and the avatar would respond with a response inside a speech bubble as well as a text-to-speech response. The avatar had a variety of different animations which accompanied each interaction with the user; it was even possible to make it dance.
2. Unity 3D environment- if augmented reality was not an option, the user could interact with the avatar in a fake environment in Unity. This environment weather was also dependant on the current location of the user; if it was snowing, ground would have a snow texture and the clouds in the environment would be producing snow particles. This function included: wind speed, cloud density and air moisture (mist)
3. Simple texting interface- the user could also use a simple text-message environment to interact with the chatbot, which was aimed to users who were less in touch with technology or wanted a quick response.

I was also responsible for handling generating all user interface scenes, language translation, user settings, message passing and integration of all the different APIs used in the project. Overall, the project was successful, and the showcase video can be found here

One project I am particularly proud of was my 4th year Fundamentals of Modelling and Rendering coursework which involved creating a Monte-Carlo recursive raytracer from complete scratch in pure C++. The raytracer had to be powerful enough to calculate: Blinn-Phong lighting, textures, shadows, reflections and refractions.

The user would load in a model and the raytracer would calculate the colour of every pixel by shooting a ray into the environment depending on the current rotation and position of the virtual camera. If the ray interacted with an object, it would calculate the colour values of the surface (with Blinn-Phong light and texturing) and would also shoot off more rays into the environment from the surface. The direction of each the new ray was calculated using the Monte-Carlo method. Once a max threshold of bounces were reached, all the colour values were summed together by recursively tracing every ray and multiplying by its impact on the final colour. The final result was very interesting and the raytracer created much better images than simple OpenGL.