This post will discuss some existing games which employ machine learning techniques and, as much as possible, how they do it.
1.0 – Creatures
Creatures is a video game series which came out in the 1990’s and uses neural networks and biochemistry to create artificial life. The creatures in the game, called Norns, are taught how to act and live by the players through interaction and breed using simulated DNA.
1.1 – Neural Network & Learning
The neural network used for each creatures was designed to be efficient and dynamic. The network can be mutated and recombined during reproduction while still maintaining a good, if not better, level of function. Grand and Cliff define many different types of input data as well as 6 parameters for each neuron: Input types, Input gain, Rest State, Relaxation Rate, Threshold (output remains zero until threshold is passed), and State-Variable Rule (used to compute the value of the neuron from one or more input signals) (Grand, and Cliff, 1998). The neurons are grouped into different lobes which perform various decision-making and logic tasks to control the creatures behaviour. The machine learning really comes in in the concept and decision lobes. The concept lobe contain neurons which ‘watch’ 4 input signals from the creatures sensory system. These neurons fire when all 4 inputs are activated, and are given new connections when all 4 drop to zero. Another level of algorithms attempts to keep a pool of repeatedly firing neurons connected for a long time while also ensuring that there are available neurons to be committed to new connections (Grand, and Cliff, 1998). The Decision layer then has one cell for each possible action, and these cells have many connections to signals from the concept layer. both positive and negative signals come in and are summed by the cell. The decision cell with the highest value is taken as the choice of action by the creature. These signals also retain a susceptibility based on their current influence and are adjusted if positive or negative feedback is received. Of course, these connections can also be deemed influential to the connected action, and will seek out new sources of input from the concept layer (Grand, and Cliff, 1998).
1.2 – DNA & Reproduction
Each creature has a genome which defines the way it looks as well as influencing some of the internal structure of the brain and decision process. When creatures are bred, the genes of both parents are spliced and mixed together with a small amount of random mutation (Grand, and Cliff, 1998). Although this process isn’t exactly machine learning, the process does require each creature to live past puberty, and allows the player to keep favorable traits alive and produce creatures which model their parents both visually and behaviorally.
2.0 – Black & White
In Black & White the player acts as a god, gathering followers and helping them survive and thrive. At the beginning of the game, the player selects a creature who will help them along the way. The creature learns how the player plays and constructs decision trees based on feedback from the player and observations of the world (Wexeler, 2002). Details on the implementation of the creature AI in Black & White is scarce, but I gather that a very simple neural network was used in conjunction with the aforementioned decision trees to create the behaviour (Champandard, 2007).
3.0 – Forza Motorsport
As mentioned in my post on ML for player modelling, the Forza Motorsport series implements machine learning in their Drivatar™ technology.
4.0 – City Conquest
City Conquest is a Kickstarter game that was successfully funded on April 29, 2012. There is not a whole lot of documentation on this game or how machine learning was used, but an interview with the games designer and engineer, Paul Tozour, provides useful insight into the system that he calls The Evolver.
As a tower-defense-style game, City Defense requires a lot of playing and tuning to ensure that the cost of each building/unit is fair and that there are no gameplay strategies or loopholes that can cause one player to greatly over- or under-power the other. The Evolver is a system which uses randomly generated script to run game matches in an evolutionary tournament. Each script defines the order or purchase and placement of buildings in the game. After an initial tournament of 400 scripts, The Evolver uses simple processes to breed/combine successful scripts and add in random mutations to the scripts as well. Tozour could run the Evlover for one or two days straight to try and discover the best strategy in the game. Once the script was found, the designers could play it back and see why the methode came out on top. Tozour noted that although this method doesn’t replace human gameplay testing, it provides a great development resource at a reasonably low cost that expedites their tuning process greatly (Champandard, 2012).
5.0 –
References
Grand, Steven, and Dave Cliff. “Creatures: Entertainment Software Agents with Artificial Life.” Autonomous Agents and Multi-Agent Systems 1: 39-57. SpringerLink. Web. 9 July 2014.
J. Champandard, Alex . “Evolving with Creatures’ AI: 15 Tricks to Mutate into Your Own Game.” . AMGameDev.com, 1 Oct. 2007. Web. 10 July 2014. <http://aigamedev.com/open/review/creatures-ai/>.
Wexler, James. “Artificial Intelligence in Games.” Rochester: University of Rochester (2002).
J. Champandard, Alex. “Top 10 Most Influential AI Games.” Your Online Hub for Game/AI. AIGameDev.com, 12 Sept. 2007. Web. 11 July 2014. <http://aigamedev.com/open/highlights/top-ai-games/>.
J. Champandard, Alex. “Making Designers Obsolete? Evolution in Game Design” Your Online Hub for Game/AI. AIGameDev.com, 6 Feb. 2012. Web. 13 July 2014. <http://aigamedev.com/open/interview/evolution-in-cityconquest/>.