Can you spot the bot in the video clips below (spoilers at the end of the page)?

This project is my master’s thesis, “Using recorded player input to train machine learning agents to test-play digital worlds”.

Inexperienced teams commonly have issues with a proper quality assurance (QA), or software verification (SV), pipeline. Many (rare) bugs will frequently be out of QA, or SV, testing scope, even for large developers. Some bugs only occur in rare edge cases that require a multitude of play tests of similar-but-not-identical repeats of inputs to find. Discovering these rare bugs is a tedious task that requires a lot of time and manpower, which small teams may be short on. An agent that can automatically perform this task by repeatedly running play-tests with slightly-altered inputs might be an effective solution to this problem. This allows small or inexperienced teams to thoroughly test their games with minimal manual work and only a small set of training data. Because a minimal set of training data is required, this also addresses another QA issue, which is concise reproduction steps. If this research is successful then it will enable development teams to develop training sets for testing agents with minimal effort.
Video clip showing the prototype I made and used for my research:

Developing quality video games is a complex and difficult process involving several different fields of expertise. Game balance and play testing are highly integrated processes that together present an open-ended challenge for developers. My research explores new methods to streamline the early stages of game development and testing through machine learning agents. The tools provided by Unity’s Machine Learning Toolkit used was not yet production-ready during my study (for more information, see the bug reports included in the paper linked below), but I expect that these tools will mature and improve as more use-cases are found. I wish to encourage future researchers and game developers to look for ways to enhance their workflows with machine learning agents.

The result I found is promising for further study. It may be possible to create an agent that is indistinguishable from human play. The result is inconclusive because the sample size was too small. Below is the result I gathered using a Twitter poll, my thesis contains more information for interested readers.

Link to research poster, as inspired by “How to create a better research poster in less time (#betterposter generation 2).” on YouTube.
Link to paper.

Any questions or feedback about the paper are more than welcome, please contact me through any of the below platforms:

Spoiler:

Video clip A is a person playing, video clip B is a bot playing.