Tic Tac Toe: Unit Tests

One of the biggest investments in time/money you have as a game designer goes into doing graphics and user interfaces. It’s important to check out the game logic before putting a bunch of UI into place. The UI is more difficult to debug than the game logic is, so it’s a good thing to make sure your game logic is solid and functional before trying to plug it into the UI. That eliminates a possible class of bugs and allows you to debug faster. The game logic should be independently testable without any user input. This is an ideal place to create unit tests.

Unit tests can be set up to check every single possible state of a function. Apple has a built in testing framework called XCTest. This framework allows boolean comparisons, value comparisons, and whether a function properly throws an error. There is a guide to writing good unit tests in Xcode here.

The reason I made all of these stand alone functions was because it would make them easier to unit test. Instead of having to create a View object and call these methods on those objects, I can just call functions and pass them state to test each condition within them.

I don’t want to repeat all of the code I already wrote that I am testing, so if you want to refer back to what I am testing, check out my previous post here.

Testing Reset Board

The smallest and easiest function to test is the function that creates the array that holds all of the tile state. This function has no parameters and returns one object that will consistently be the same no matter what.

Here is the test function for resetBoard():

func testResetBoard() {
	let accurateBoard:[TileState] = [TileState.notSelected,
					 TileState.notSelected,
					 TileState.notSelected,
					 TileState.notSelected,
					 TileState.notSelected,
				         TileState.notSelected,
					 TileState.notSelected,
					 TileState.notSelected,
					 TileState.notSelected]
		
	let testBoard = resetBoard()
		
	XCTAssertEqual(accurateBoard, testBoard)
}

The first variable of this test is to create an instance of what I expect the board to look like. I didn’t use a for loop to create the board because that is the code I am testing within my function. If I use the same code to create my test cases, then the tests will never fail, but the code still may not work the way I expect.

The next variable is generating a board using the resetBoard() function.

Finally, I am running an XCTest function to verify that the two variables are the same. XCTest has functions to assert equality and inequality. I am checking that they have the same capacity and hold the same data types. I ran the test and the test passed, so I know that my resetBoard() function will give me an array of nine TileState.notSelected instances.

Testing Switch Player

The switchPlayer function is slightly more involved than the resetBoard() function. The GameState enum contains three possible cases that need to be tested:

  • The current player is Player A
  • The current player is Player B
  • The game hasn’t started yet and no one is the current player.

Each of these cases will need to be tested to ensure that this function works properly. First, I want to create instances of Player A and Player B to plug into my tests. I could just create them within the function call, but I like to have labels for things:

func testSwitchPlayer() {
	// Player Objects
	let playerA = GameState.playerA
	let playerB = GameState.playerB

The first case I will test is what happens when Player A is the current player:

// Player A Case
let firstPlayerTest = switchPlayer(currentPlayer: playerA)
XCTAssertEqual(firstPlayerTest, playerB)

According to our function logic, if Player A is the current player, then switching the player should return an instance of Player B. Likewise, starting with Player B should return an instance of Player A:

// Player B Case
let secondPlayerTest = switchPlayer(currentPlayer: playerB)
XCTAssertEqual(secondPlayerTest, playerA)

The final test case is what happens where there is no current player. I determined in my logic that Player A would always start, so if we are switching the current player at the beginning of the game, Player A should always be the returned value:

	// Not Currently Playing Case
	let thirdPlayerTest = switchPlayer(
            currentPlayer: GameState.notPlaying)
	XCTAssertEqual(thirdPlayerTest, playerA)
}

Again, this test passes and I have verified each possible test case.

Testing Make Move

One reason I am testing these in order of complexity is so that I can use some of these tested methods within my future tests. For the makeMove() function, I need a lot of instances of a game board. I don’t want to have to type out all of these boards like a monkey, so being able to just generate a board using my resetBoard() function is very useful. I feel comfortable using that for this because I have tested it and verified it works.

First I want to create some reusable objects for setting up my assertions:

// Objects
let playerA = GameState.playerA
let playerB = GameState.playerB
let notPlaying = GameState.notPlaying

let selectedA = TileState.playerA
let selectedB = TileState.playerB
		
let blankBoard = resetBoard()

I have to create a .notPlaying test case even though I do not believe there will be a point in my code where it actually gets used. It is for the sake of safety and completeness that I include it and I need to test it to verify that it’s benign if it does ever wind up getting used.

First I want to check what happens if Player A makes a move:


// Player A move
let firstNewBoard = makeMove(tile: 0, currentPlayer: playerA, board: blankBoard)
XCTAssertNotEqual(blankBoard, firstNewBoard)
XCTAssertEqual(firstNewBoard[0], TileState.playerA)
XCTAssertEqual(firstNewBoard[5], blankBoard[5])

Since makeMove() returns a new array of TileState, our blankBoard is never mutated and can be used as a reference point against any board that is generated by makeMove(). Since I have generated a new board called firstNewBoard that should be different from blankBoard, I want to verify that they are in fact different. I also want to verify that the item I mutated in the function call actually changed to what I expect. Finally, I want to verify that nothing else changed. I didn’t write a test case for every element, so I chose one to test. I feel this is good enough and I don’t really want to write a billion tests around this when I feel reasonably sure this works as it should with a random sample. Next, I want to do the same using Player B:

// Player B move
let secondNewBoard = makeMove(tile: 2, currentPlayer: playerB, board: blankBoard)
XCTAssertNotEqual(blankBoard, secondNewBoard)
XCTAssertEqual(secondNewBoard[2], TileState.playerB)
XCTAssertEqual(secondNewBoard[8], blankBoard[8])

This code looks substantially similar to the Player A code, but I did want to change the tiles with the moves and the random check just so I wouldn’t accidentally have an edge case that happened to pass my tests. I have done this before and it makes you feel rather silly.

So far I have tested both Player A and Player B with a blank board. I would like to now write a few tests that simulate an actual game to verify that this works out of isolation:

// Moves by both Player A and Player B
let firstMove = makeMove(tile: 4, currentPlayer: playerA, board: blankBoard)
let secondMove = makeMove(tile: 0, currentPlayer: playerB, board: firstMove)
let thirdMove = makeMove(tile: 2, currentPlayer: playerA, board: secondMove)
		
XCTAssertNotEqual(firstMove, blankBoard)
XCTAssertNotEqual(secondMove, firstMove)
XCTAssertNotEqual(thirdMove, secondMove)
		
XCTAssertEqual(thirdMove[4], selectedA)
XCTAssertEqual(thirdMove[0], selectedB)
XCTAssertEqual(thirdMove[2], selectedA)

I simulated a game through the first three moves. I used the previous move’s result as the parameter for the next move’s function. The tests I wrote were to verify that the board does in fact change after each method call. I am also running tests on the final board to verify that a move made in the first board continues to be passed down through each successive move.

Finally I am adding my sanity check that if I am passing in a not playing state that the board does not change:

	// Not Playing move
	let notSelectedBoard = makeMove(tile: 5, currentPlayer: notPlaying, board: blankBoard)
	XCTAssertEqual(notSelectedBoard, blankBoard)
}

I am not testing this case as rigorously because I don’t anticipate it ever happening. Also, if it does, it shouldn’t really change much of anything.

This test also passed without issue. (Note: I am writing this blog post and my tests in parallel. Had something failed, I would have left the previous code as is and would include the test failure and explain why. These tests have been passing because the functions were designed very simply, making it easier to write good test cases, which was kind of the point going in.)

Test Win Game

I’m finally testing my monster function that includes all the various ways a turn can end. I am sure everyone is thrilled about reading two thousand words on doing repetitive unit testing logic, so I broke my tests up on this method to allow me to explain them without repeating myself a billion times.

First I will check for the diagonal win conditions, mainly because there are only two of them and that gives you the general idea about how the rest of the tests would be set up for vertical and horizontal wins. Again, I start with my game objects:

// Objects
let playerA = GameState.playerA
let playerB = GameState.playerB
		
let playerAWin = GameEndState.playerAWin
let playerBWin = GameEndState.playerBWin
		
let blankBoard = resetBoard()

The checkForWin() function returns an optional game end state. It’s optional because it’s possible that there are still moves to be make and no one has won yet. Since I am checking for wins here, I made objects for win conditions. When I test for other conditions I will create different objects.

Here are my tests for an upper left diagonal win:

// Check upper left win
let upperLeft1 = makeMove(tile: 0, currentPlayer: playerA, board: blankBoard)
let upperLeft2 = makeMove(tile: 4, currentPlayer: playerA, board: upperLeft1)
let upperLeft3 = makeMove(tile: 8, currentPlayer: playerA, board: upperLeft2)
		
let checkForUpperLeftWin = checkForWin(currentPlayer: playerA, board: upperLeft3)
let notAWinYetUpperLeft = checkForWin(currentPlayer: playerA, board: upperLeft2)
let wrongPlayerUpperLeft = checkForWin(currentPlayer: playerB, board: upperLeft3)
		
XCTAssertEqual(checkForUpperLeftWin, playerAWin)
XCTAssertNil(notAWinYetUpperLeft)
XCTAssertNil(wrongPlayerUpperLeft)

I generated a not quite accurate game board by simply giving Player A a win without letting Player B have any turns because I am a bad person. I look at this data and see a few places we can test to verify that this code works. We test the actual win condition to ensure that a win is returned. I also tested the move before the win to verify that the GameEndState returns nil. Finally, we are checking whether the current player has won or not, not if a win exists on the board. Even though there should be a win on the board, it is not for Player B, so this should come back nil as well.

I did the same code pretty much for the upper right win case and ran the test. It passed, so now I am moving on to testing for a draw.

A draw uses some different cases from our state objects than a win, so I need to create some new object instances:

// Objects
let playerA = GameState.playerA
let draw = GameEndState.draw
		
let selectedA = TileState.playerA
let selectedB = TileState.playerB
let notSelected = TileState.notSelected

Since I don’t really care about checking for wins, I just created one player instance for game state, but more importantly I included the draw state that I didn’t include in the last batch of tests. I created instances of tile state so that I could easily construct boards to test for various conditions:

// Boards
let drawBoard:[TileState] = [selectedA, selectedA, selectedB,
			     selectedB, selectedA, selectedB,
			     selectedA, selectedB, selectedB]
let blankBoard = resetBoard()
let nonBlankBoard:[TileState] = [notSelected, notSelected, notSelected,
				 notSelected, selectedA, notSelected,
				 notSelected, notSelected, notSelected]

(Trying to purposely construct a non-winning draw board is kind of a pain in the butt!)

I want to test three different scenarios:

  • There is a draw
  • The game has just begun
  • There has only been one move

One cardinal rule of unit testing is to test for failure conditions. In this context, anything where the game should continue is a failure condition. I am not checking with a win here because I am already checking for that in my other unit tests.

Here are my final tests to verify my draw condition works:

// Test variables and assertions
let drawCondition = checkForWin(currentPlayer: playerA, board: drawBoard)
let startGame = checkForWin(currentPlayer: playerA, board: blankBoard)
let firstMove = checkForWin(currentPlayer: playerA, board: nonBlankBoard)
		
XCTAssertEqual(drawCondition, draw)
XCTAssertNil(startGame)
XCTAssertNil(firstMove)

The draw tests also passed. I originally was going to test to ensure that the game continues if there isn’t a draw or a win, but that functionality is already being covered in both of those unit tests, so I don’t feel I need to create an entire function around it.

Ensuring Code Coverage

I’ve written what seems like an excessive number of unit tests, but how do I know I actually covered every failure case?

Starting in Xcode 7, Apple introduced a code coverage feature that is able to generate a report telling you how much of your code base is being touched by unit tests.

To enable this feature, you need to edit your target scheme:

Next, choose the option in Debug to enable code coverage data to be gathered:

Be sure to show the test bundle and check the code covered by the test bundle rather than the one that shows up by default. In this screenshot you see that the top bundle shows 0% test coverage of the State Utils class when that is basically the only thing being tested so far in the application.

Conclusions

The sample code so far for this project can be found here.

I know this wasn’t the most exciting post in the world, but there are a few take aways from this. One reason these tests were so easy to write was because I designed the code to be easy to test. How many of us have gone to work on a legacy code project that is impossible to update because there are vast multitudes of side effects and something completely unrelated to your feature breaks because of unnecessary dependencies?

I had some concerns about my very large function checking for win conditions and how I feel comfortable that my conditional logic will work as I intend. When I begin plugging this into the UI, if something isn’t working properly I will know that it’s an issue with the UI code and not with my game logic.

Speaking of UI, my next few posts will revolve around setting up the user interface and the interactive components of my game. I will be treating these separate from my game logic and as self contained units of functionality. Stay tuned!

Tic Tac Toe: Rules and Conditional Logic

The next bit that I am doing in my Tic Tac Toe game is putting in place the game rules and conditional logic around the actual game mechanics. I am saving the UI code for another time because that will be far more intensive than laying out the rules.

One of my goals with my code is to make it completely testable. One way of doing that is to create stand alone functions that can be tested outside of the game scene. Rather than creating a large nest of mutating side effects, I would like to take in parameters and return parameters, thus keeping all of my changes safely within a testable function. This will likely go through some refactoring in the future as I think about the best way of doing things. I know when I tried this before I had a lot of people yelling at me about the way I chose to write my code. It’s my code and my codebase, so I will write it however I please.

Board Properties

Tic Tac Toe is a fairly simple game. It is a board of nine squares. It has two players. On each player’s turn, they choose a square that has not been chosen yet. If there are three squares in a row, that player wins. If there are not, then the next player goes. This continues until either someone wins or all of the squares have been chosen.

This means that we need to have some kind of structure to represent the board state and we need to keep track of the current player:

var board:[TileState] = []
var currentPlayer:GameState = .notPlaying

I’m opting to use an array of tiles to represent the tile state. If this were a more complex board I would opt to do a different data structure and lay out actual rows and columns. However, since there are only nine tiles to keep track of and I am planning to number them when I set up the UI, I am opting for the more simple route and hard coding my logic. I know this is making people pull their hair out and scream, but I want to try and keep things simple. The rules haven’t changed in however many years this has existed, so the likelihood I will need to update requirements is minimal. I may change my mind, but that is for refactoring, not prototyping.

I also need to have a mutating variable to track the state of who is the current player. Since the game begins without a current player, the state is initialized as .notPlaying.

Setting Up the Board

The board is represented by a set of nine tiles. These tiles will be selected by the two different players, but not initially. When the board is initially laid out, it will be completely unselected. This can be represented by a function:

func resetBoard() -> [TileState] {
	var board = [TileState]()
	
	for _ in 0...8 {
		let tile = TileState.notSelected
		board.append(tile)
	}
	
	return board
}

This function returns an array of nine tiles that are all in the unselected state. These can be accessed by their index, which is how I will make moves and how I will check for wins.

Switch Players

When I was going to school for programming, my teacher told us that if you are describing what a function does and you have to use the word “and,” then you need another function. Each function should do one and only one thing. These functions can call other functions that take care of that functionality for you, but a function should be an encapsulated piece of code that does one thing.

In that spirit, I am adding a function to switch the current player:

func switchPlayer(currentPlayer: GameState) -> GameState {
	switch currentPlayer {
	case .playerA:
		return GameState.playerB
	case .playerB:
		return GameState.playerA
	case .notPlaying:
		return GameState.playerA
	}
}

I am making the determination in this game that Player A always goes first. You can do Rock Paper Scissors Lizard Spock to determine which player is Player A, but within my game logic Player A always goes first. In my exhaustive switch statement, I am checking to see what the current game state is. If the current player is Player A, then it is now Player B’s turn. If the current player is Player B, then it’s Player A’s turn. If this is the beginning of the game and the current state is Not Playing, then by default it will be Player A’s turn because Player A always goes first.

Making a Move

Beyond the UI, what goes into making a move? Making a move means selecting one tile within the array of tiles and changing its state to reflect which player you are. To keep this functional, you need to take in a board state, a current player state, and the index of the tile whose state needs to change. You then return a brand new board state based on the changes taken in by the function:

func makeMove(tile: Int, currentPlayer: GameState, board: [TileState]) -> [TileState] {
	var newBoard = board
	
	switch currentPlayer {
	case .playerA:
		newBoard[tile] = .playerA
	case .playerB:
		newBoard[tile] = .playerB
	case .notPlaying:
		newBoard[tile] = .notSelected
	}
	
	return newBoard
}

I do not believe that the final .notPlaying state will ever be called, but to ensure an exhaustive switch statement and to make sure the application doesn’t crash, it seemed logical to just include that as a case. It would not impact the game in any way to just keep the tile unselected in case something goes off the rails.

Checking for a Win

The final stand alone function I want to create is the function that checks for a win. As anyone over the age of six can tell you, most Tic Tac Toe games end in a draw. So when checking for a win, you must also consider the possibility there are simply no more moves to be made and the game ends unceremoniously.

To this end, we need another state enum to check for ending the game:

enum GameEndState {
	case playerAWin
	case playerBWin
	case draw
}

The game can end in one of three states:

  • Player A wins
  • Player B wins
  • Stalemate

To check for a win, you need to know the current player and the current state of the board. Since it’s possible that the game hasn’t entered one of these states yet, you need to make the return type optional:

func checkForWin(currentPlayer: GameState, board: [TileState]) -> GameEndState? {
	var requiredState:TileState
	var gameEnd:GameEndState
	
	if currentPlayer == .playerA {
		requiredState = .playerA
		gameEnd = .playerAWin
	} else if currentPlayer == .playerB {
		requiredState = .playerB
		gameEnd = .playerBWin
	} else {
		return nil
	}

In order to satisfy the compiler’s type safety, you need to create a conversion between the current player being passed in and the states you are checking. You can’t just compare the board’s TileState directly against the current player because they are of different types even though their members have the same name.

There are eight possible win states in a game of Tic Tac Toe:

  • Three across the top
  • Three across the middle
  • Three across the bottom
  • Three down the left
  • Three down the middle
  • Three down the right
  • Three diagonally from the upper left
  • Three diagonally from the upper right

I am checking each of these based on the index of the position in the array of tiles. If you count the tiles starting from the upper left and continuing to the right and down, they have a predictable index. The upper left tile is represented as index 0 and the lower right tile is represented by index 8. We can use the indices to check for the various win conditions. Let’s start with the across win checks.

	// Three across the top
	if board[0] == requiredState && board[1] == requiredState && board[2] == requiredState {
		return gameEnd
	}
	
	// Three across the middle
	if board[3] == requiredState && board[4] == requiredState && board[5] == requiredState {
		return gameEnd
	}
	
	// Three across the bottom
	if board[6] == requiredState && board[7] == requiredState && board[8] == requiredState {
		return gameEnd
	}

These checks are checking for contiguous tiles that all match the current player state. If any of these conditions are satisfied, the current player wins the game and the rest of the logic is never executed as it’s unnecessary. Next, we do something similar with the vertical win conditions.

	// Three down the left
	if board[0] == requiredState && board[3] == requiredState && board[6] == requiredState {
		return gameEnd
	}
	
	// Three down the middle
	if board[1] == requiredState && board[4] == requiredState && board[7] == requiredState {
		return gameEnd
	}
	
	// Three down the right
	if board[2] == requiredState && board[5] == requiredState && board[8] == requiredState {
		return gameEnd
	}

These are slightly more complicated as the indices are not contiguous. I made a diagram of the correlating tiles to ensure that my hard coded values were correct. Yes, I know hard coded values are evil. Bite me.

Finally, we check for the two diagonal win conditions.

	// Three diagonally from the upper left
	if board[0] == requiredState && board[4] == requiredState && board[8] == requiredState {
		return gameEnd
	}
	
	// Three diagonally from the upper right
	if board[2] == requiredState && board[4] == requiredState && board[6] == requiredState {
		return gameEnd
	}

If none of these win conditions have been satisfied, that means that the board is in one of two states. Either we are still playing and the game should continue, or there are no more moves to be made an no one can win. The easiest way to check for a continuing game is to check if there are any unselected tiles still on the board:

	// Draw
	for tile in board {
		if tile == .notSelected {
			return nil
		}
	}
	
	gameEnd = .draw
	return gameEnd
}

If there is even one tile that isn’t selected, the game continues. By making the game end state optional, we are allowing a scenario where you return nil if the game is not in fact at its end state.

If you have exhausted all of these options, where there isn’t a win state and there are no more tiles to be selected, then the game has concluded in a draw. You set the game end state to a draw and return that at the very end if there has not been any scenario where the method can end differently.

By far the largest and most involved function I have in this application is the checking for win function. I am not super happy with how long this turned out, but for the time being I can’t really think of a better way of doing this. There is a lot of conditional logic in here that is specific to the game, so there is going to be a lot of code, I just don’t know if there should be this much. I will ponder in the future if there are ways of cutting down on this.

Conclusion

You can access the current version of the code here.

This isn’t necessarily the way someone else would set up their application, but there isn’t a right or a wrong way to do anything. There are matters of personal preference. There is probably a way to make this code less complex, which I hope to figure out later. One thing that frustrates me about talking to other people about code is this idea that code is written in stone and can never be changed. This isn’t the case at all.

It’s better to just get something done and to refactor it later. Otherwise, you stare at a screen and nothing gets done. When I worked for Brad Larson on our rewrite of his robotics software, he regularly refactored his code. He would notice he was doing something over and over again and that it could be pulled out as a stand alone function. The process of writing the code base informed his refactoring decisions.

You can’t just go into a code base knowing all the challenges you will encounter. You learn that by working with it. My code will probably look significantly different than this when it’s done because I will figure out better ways of doing things and I will refactor the code base. This is how you should be approaching your code.

So this is all well and good to have these disparate stand alone functions, but how can we tell if they actually work or not? In my next blog post, I will set up unit tests to prove that these work as designed.

Augmented Reality: ARKit and SceneKit

One of the most exciting new frameworks announced at WWDC 2017 was ARKit. Virtual reality has been fairly hot the last few years, but in my opinion augmented reality has a better future. Both are garnering a lot of attention so I am writing this post about what augmented reality is and the easiest way to natively integrate it into your applications.

What is Augmented Reality?

Virtual reality is the creation of a totally immersive experience. Think of the holodeck from Star Trek. The characters enter a room and the computer creates a completely immersive simulation. The characters can explore cities and planets or go back in time. Like the TARDIS, the holodeck seems bigger on the inside. It theoretically is indistinguishable from the real world.

Stepping back in time virtually.

Augmented reality is different. Augmented reality doesn’t seek to immerse you in another world, but rather exists to add to the world around you. The best known example of this is Pokemon Go. Pokemon Go overlays Pokemon characters within the real world to emulate the experience of being a Pokemon trainer and going out to capture Pokemon. This allows players to experience what it would be like to live within the game.

This distinction is one reason I feel AR has a brighter future than VR. One reason that people got into Harry Potter is partially embedded in the idea that this is a hidden aspect of the real world. It’s the idea that somewhere in London you can find a hidden neighborhood where you can buy magic wands or that you could receive an acceptance letter to Hogwarts. It’s a distinction between total escapism and having the idea that there is magic hidden within the real world.

Augmented reality was possible on the iPhone before ARKit was introduced. I found a book on how to implement AR going back to iOS 5. Pokemon Go predates ARKit. What ARKit does is wrap a lot of complexity within a few easy abstracted methods and classes.

Augmented reality requires three different functionalities:

  • Tracking
  • Registration
  • Visualization

Tracking is the ability of the application to detect surfaces and know what objects exist in space. In Pokemon Go, tracking is used to ensure a Pokemon appears on a road or sidewalk as opposed to inside of a tree.

Registration catalogs and keeps track of areas and objects of interest. In the Pokemon Go example, registration is used to keep track of which Pokemon exist in the area and where those Pokemon are supposed to appear. If you move your phone away from the Pokemon and then come back to it, the Pokemon should still be there, unless someone else captured it.

Visualization encompasses everything that you see in the scene. This is where the Pokemon’s mesh is loaded, shaded, and rendered. This is the main point in the process that people are most aware of because it’s the thing they see directly.

What Does ARKit Do For You?

ARKit is not a very large framework. It has fewer than ten classes and those classes do not have a lot of methods in them. The reason for this is that ARKit only handles two of the three requirements for augmented reality: Tracking and registration.

Tracking and registration are pretty straightforward conceptually without a lot of special modifications. Nothing about these tasks are going to fundamentally change very much between applications. An AR measuring tape is going to function the same way in respect to these tasks as a game or a portal to another dimension. That makes it a prime candidate for a Cocoa framework. It wraps a lot of boilerplate code in some friendly and approachable Swift code.

This small framework is deceptive in how much it actually does. The vast amounts of machine vision being done under the hood is astonishing. If you were responsible for setting up the real time video analysis and image filtering for object detection on your own, it would take a research team and five years.

What ARKit Doesn’t Do For You

ARKit does not handle the most visible step in the process, which is visualization. Just knowing the ten or so classes in ARKit doesn’t really get you a working augmented reality application. This is similar to learning CoreML. Both of these frameworks wrap a bunch of boilerplate code around a complex but relatively similar pipeline so that you don’t have to worry about the ins and outs of how the camera is capturing frames. You just have to worry about telling it what to do once the frame is captured.

ARKit in a Nutshell

I mentioned earlier in this post that ARKit isn’t a very large framework. There are only three broad categories of object you need to be familiar with. Within these three categories are three classes, so about nine classes total.

ARKit Foundational Structures

The first category of object are what I call ARKit’s foundational structures. These are objects that manage ARKit in your application and talk to other things on your behalf.

These classes are:

  • ARSession
  • ARConfiguration
  • ARSCNView/ARSKView

If you’ve done any work with AVFoundation, you should be familiar with sessions. ARSession is a shared object that manages the camera and motion processing needed for augmented reality experiences. You can’t have an AR application without an ARSession. It’s the project manager of the AR functionality.

ARConfiguration is an abstract base class that has a few different subclasses depending upon what you need from your application. ARWorldTrackingConfiguration is the most comprehensive and highest quality tracking configuration. It allows plane detection and hit testing. Not all devices support world tracking. For those devices you need to offer AROrientationTrackingConfiguration, a more limited configuration. Finally, If you’re only working with facial data, you would use ARFaceTrackingConfiguration.

The final set of classes are the views. Both ARSCNView and ARSKView inherit from their respective 2D or 3D frameworks. They are subclassed to respond to ARKit events like hit testing. You will need to create an instance of the ARSCNView at the top of your class:

var sceneView: ARSCNView!

If you create your application from the Augmented Reality template it will automatically configure the session for you:

override func viewWillAppear(_ animated: Bool) {
	super.viewWillAppear(animated)
		
	// Create a session configuration
	let configuration = ARWorldTrackingConfiguration()

	// Run the view's session
	sceneView.session.run(configuration)
}

The session has to be set to both run and pause. The pause is set when the view disappears:

override func viewWillDisappear(_ animated: Bool) {
	super.viewWillDisappear(animated)
		
	// Pause the view's session
	sceneView.session.pause()
}

Now that a session is in place, you need to add some ARKit objects.

ARKit Real World Representations

In order to place virtual objects in the real world, you need a way to understand where in three dimensional space the object should be rendered. Our eyes and brains recognize objects and can remember where books and pictures are when we leave the room. The program has to be able to do the same thing. There are three main objects ARKit uses to analyze the world and persist objects:

  • ARAnchor
  • ARPlaneAnchor
  • ARHitTest

ARAnchor instances represent real points in space. They are used for placing objects in the augmented reality scene. If you wanted to place a virtual pug in the middle of a rug, you would create an anchor in space to tell the renderer where to draw the pug. ARPlaneAnchor is similar but deals exclusively with flat surfaces.

ARHitTest instances are information about a real-world surface found by examining a point in the device camera view of an AR session. These include:

  • Feature Points
  • Estimated Horizon Plane
  • Existing Plane

If you want more information about these types, you can access it here.

Camera and AVFoundation Wrappers

One difficult thing in iOS development is pulling camera frames from the camera and doing something with them. If you’ve ever mucked around with AVFoundation, you know what a pain in the neck it is. A lot of classes are incredibly long and have very similar names. ARKit takes these tasks and abstracts them away from you so you don’t have to worry about them. These classes are:

  • ARFrame
  • ARCamera
  • ARLightEstimate

The ARFrame is a video image and position tracking information captured as part of an AR session. Properties you can muck around with on this object include the depth data and the time stamp. The captured image is a CVPixelBuffer which allows you to process this as you would a frame of video (because that’s exactly what it is).

The ARCamera contains information about the camera position and imaging characteristics for a captured video frame in an AR session. This includes the tracking state and the orientation of the camera. You also have access to the camera geometry, which allows you to perform matrix transforms.

Finally, the ARLightEstimate estimates scene lighting information associated with a captured video frame in an AR session. This primarily involves the intensity and color temperature. You can use these to incorporate into your lighting shaders to get the virtual objects to match the ambient lighting in the scene.

By this point, you’ve basically covered everything you need to understand about ARKit, but you still don’t see anything on the screen. From this point, you will need to work with a renderer.

ARKit with SceneKit as the Renderer

ARKit has several options for both native and non-native renderers for ARKit. Both Unity and Unreal plug into ARKit, allowing you to harness the power of those game engines to do some truly astonishing things. Alternately, you can use the native SpriteKit and SceneKit frameworks to do some cool stuff as well. SpriteKit as a renderer is somewhat limited, so for the rest of this post I will be creating the simplest SceneKit integration with ARKit that I can, which is to create a box that is anchored in space.

I used
Mohammad Azam’s
Udemy ARKit course as a basis for this sample project. I highly recommend the rest of his course if you want to get more in depth with ARKit.

Augmented Reality has its own template. It isn’t under Games.

The following code is the only code I had to add to this project. Everything else was already set up by the template. The template provides a scene already, but I want to render a box rather than the built in ship asset:

// Create a new scene
let scene = SCNScene()
		
let box = SCNBox(width: 0.2, height: 0.2, length: 0.2, chamferRadius: 0)

SceneKit has a large library of built in primitive objects. It has a surprising amount of built in functionality that allows you to make a lot of progress fairly quickly.

The box requires some surface customization. The surface texture and color is determined by it’s material property. This material can be an image that is mapped onto its surface, a color, and a degree of shininess. There is a lot of customization you can do to an object’s material property, but for our purposes we’re just making it purple for now:

	
let material = SCNMaterial()
material.diffuse.contents = UIColor.purple

We have created a box and a material, but these variables are not yet associated with one another. These are going to be properties of a SceneKit object called an SCNNode. SceneKit, like SpriteKit, is completely composed of graphs of nodes. If you want something to appear on the screen, you need to set properties on it and add it to a node:

	
let node = SCNNode()
node.geometry = box
node.geometry?.materials = [material]
node.position = SCNVector3(0, 0.1, -0.5)		
scene.rootNode.addChildNode(node)

Finally, you need to commit the scene to the view:

// Set the scene to the view
sceneView.scene = scene

This isn’t much, but it’s pretty impressive for fewer than twenty lines of code. Having spent a hundred lines of code trying to render triangles with no lighting to the screen, this is magical voodoo to me.

Current ARKit Limitations

ARKit impressed me with the degree of simplicity it was able to accomplish for the boilerplate operations around capturing frames. Like most Apple frameworks in their initial release, ARKit does have a few weak points that should be improved over time. These primarily include:

  • Bad Lighting
  • Excessive Motion
  • Limited Features

If you don’t have good lighting, it’s difficult for ARKit to pick out features in a dark room. Similarly, it’s difficult for the camera to keep track of points in space if you’re waving the camera around a lot. It’s using machine vision algorithms that don’t do well with motion blur. Additionally, as you run your ARKit application, the program will keep analyzing the scene and the points may change over time.

Currently, the only objects that ARKit works well with are flat surfaces and horizontal planes. The framework isn’t equipped to recognize three dimensional objects. You can’t create a virtual cat that will recognize a chair in a scene and hop onto it. It would recognize many flat surfaces and may wind up on a table or the TV. This may be closer to how cats behave in real life, but you probably want a little more control over your virtual cats.

Final Thoughts

You can access the sample code for this blog post here. I intend to add to this repository over time.

If you’re serious about working with augmented reality, I strongly suggest learning a rendering engine like SceneKit or Unity.

For me, the limiting factor in ARKit is the same one I have for SpriteKit and SceneKit. Programmatically you can do a lot of cool stuff, but unless you have access to diverse and well done graphical assets, what you can do on your own is pretty limited. SceneKit compensates with its primitive library, but if I want to create a game about pugs I need to either find an artist to create those assets for me or I have to learn how to create them myself.

A lot of people are interested in AR for real world applications, like medical imaging. Myself, I am interested in using it to create immersive real world experiences that you can’t get any other way. I feel that this could be used as a tool for engagement. Rather than staring at our phones while blocking out the world around us, we can use them as a way to more deeply explore the world outside our devices.

Tic Tac Toe: Game State Setup

One concept that I find interesting about game development is the concept of state. When I first got interested in OpenGL, someone told me that OpenGL is simply a “state machine.” I didn’t really grasp what he meant by that, so it’s something I wanted to make sure I had a good handle on by working through these projects.

It’s a concept that I am sure is fairly straightforward but I have never heard it explained super well. Basically, in games there are aspects of the game that change. A simple example of state is if you’re playing a multiplayer game. It can only be one player’s turn at a time. That means that if you are programming a game where people take turns, you need a way to keep track of whose turn it is at any given point in time. That is your state. Additionally, it’s possible that it’s no one’s turn if the game is over. You don’t want someone to take a turn when the game has concluded. So your game has a few different states it can be at at any given point in time.

Since tic tac toe is a two-player game, I need a way to track which player is the current active player. I also need a way to indicate is the game is in a state where no one is the active player. Back in the dark ages (pre-Swift), there wasn’t a good way of tracking state, so Apple introduced GKState in their GameplayKit framework. This allowed you to build state machines to track state in your game. However, with the advent of more powerful Swift enums, it’s far simpler to just create an enum to contain and switch your state.

Creating state machines in Swift is easy and powerful. Enums are an “or” type, which means that a Swift enum can only have one and only one value at a time. Which means if you need to keep track of state, they work very well. Let’s look at the example of keeping track of the current state of the game. We’ve already determined that the game can have three different states:

  • No Game Active
  • Player A’s Turn
  • Player B’s Turn

These conditions can be directly mapped to an enum:

enum GameState {
    case notPlaying
    case playerA
    case playerB
}

This enum is named GameState, making it easy to remember what it is tracking. You can initialize this game state as a variable in your main program:


var currentPlayer:GameState = .NotPlaying

You initialize currentPlayer as not playing because this is the default state you will find yourself in until you initialize a game. Once a game begins, the currentPlayer state will change to either currentPlayer.PlayerA or currentPlayer.PlayerB. There will never be a situation where this state is blank or where there will be more than one valid case. It is exclusive.

The only other state I have to track in this simple game involves the state that any individual tile is in. The tiles can have three possible states:

  • Not Selected
  • Player A’s Tile
  • Player B’s Tile

This again maps very easily to an enum:

enum TileState {
    case notSelected
    case playerA
    case playerB
}

This takes care of all of the state I need in the game. It isn’t much, but it’s a solid start to the foundation of our game. Most of the non-UI code I need will revolve around checking and transitioning this state. You can check the code so far here. Next, I need to set up the conditional logic and rules of the game.

Game Dev Journal: Project 1 Planning Session

Begin at the beginning and go on till you come to the end; then stop.
– Lewis Carroll

My friend Brian Hogan wrote a book a few years ago called Exercises for Programmers. The gist of the book is that it gives you a list of simple exercises that get progressively harder that can be done in about an hour. One doesn’t get better at anything without targeted practice and one way to practice is to work on solving a simple problem, but doing that consistently every single day.

Many of his challenges can be done simply in a few minutes, but you can add complexity to them, especially with iOS. If you write unit tests around your solution and create a UI for them that requires AutoLayout, you add significant complexity to the challenge. This also treats the challenge as you would a project you would create in the real world. Instead of just getting practice with simple conditional logic in something like Swift, you are getting practice working with Apple’s tools and aspects of the UI that you will only create once in a real project that may take a year to ship.

I am taking this approach with my first foray into games. The simplest game I can think of to create is tic-tac-toe. I worked through a C Plus Plus book that implements tic-tac-toe as one of the first projects and does so in about a hundred lines of code. If you’re just trying to get something done as fast as possible, then this isn’t really the best learning project.

However, I want to treat this like it’s an actual game I would ship. I want decent graphics. I want an interactive UI. I want different game scenes. There is a lot to learn to make this look like a real game. All of these nice peripheral things are features I want to include in actual apps I want to ship. By learning them on this throw away project, I should be able to develop this faster for future projects.

When learning something new, it’s important to simplify things as much as you possibly can. Constraining the number of new things you have to learn is vital if you want to make any progress. If you can narrow 50 things down to five and build on from there, by the time you get to the last ten things, you should have a solid foundation built upon repetition.

In that spirit, I have broken down creating this as a serious project into a series of achievable tasks that I intend to blog about over the next few weeks:

  • The Game Logic: The backbone of this project is setting up all the conditional game logic. This includes checking for a win and switching players. I want to write this functionally so that every aspect of the game logic can be tested.
  • The UI: In tic-tac-toe, you have nine squares that can be selected. I need to find a way to set these squares up and have them respond to user interaction. I also have to determine how to disable interaction once a square has been chosen. I also want to create animations to make the game feel more natural and responsive.
  • Game Scenes: One aspect of modern games is the idea that you have more than one game scene. You start on a main menu. You can look at a leader board. There is another scene once the game is over and you win or lose. I would like to set this up as a “real” game with all of these transitions.
  • Game AI: I want you to be able to play by yourself, so it’s necessary to program an AI to play against. This takes advantage of Apple’s Gamplay AI framework.
  • Game Actions: One thing that confounded me about SpriteKit was trying to figure out if you could use Core Animation with it. I recently discovered that much of Core Animation has been adopted as the SKActions framework. I want to animate the sprites that represent the squares in some way and this promises to make that an interesting challenge.
  • Graphics and Sounds: Even though this isn’t going to be a particularly complex game, I would like to use nice graphics and sound effects for it. Since there are a limited number of elements, this gives a good intro to thinking about what one needs to make a game look polished and not like developer art. For better or for worse, graphics are the thing people notice and it’s important to find ways to differentiate yourself from everyone else. I also studied audio engineering and understanding how to put together a unique soundtrack quickly can be a valuable skill.

I’m going to use SpriteKit for this project. My goal by the end of the year is to ship a game to the App Store in SpriteKit, so this is practice for the game I want to ship. I don’t know how many prototype games I should do in SpriteKit, but I do want to work through at least one. I know SpriteKit isn’t cross platform, but I don’t really want to deal with figuring out how to ship to another platform. I want to get more comfortable with shipping through iTunes Connect and with Swift, so it makes more sense for me personally to do this is SpriteKit instead of something like Unity. I want to get to Unity/Unreal later this year (or earlier if I am bored and want to procrastinate).

My goal over the next few weeks is to make steady progress on this project. I plan to post at least one blog post a week for this entire year. I am doing this as a nights and weekend project, so it could take a while to finish. The goal isn’t speed so much as consistency. I know that it’s important to have deadlines or else you don’t ship anything, but deadlines cause me severe amounts of anxiety. I have goals about how long I want this to take, but life happens and I don’t want to assume this can be done over a weekend if it will actually take me a month because I don’t know what I am doing.

Again, the point of this project isn’t to build a tic-tac-toe game. The purpose is to use a simple problem as a foundation for figuring out design patterns in the most straightforward way possible. This may feel like overkill for something that isn’t particularly interesting, but I think practicing on this will make the next game easier. I also believe there are probably things in this process that I don’t know I need to do until I do it, so I want to uncover all the bodies before I start on a project I care about.

Goals for 2018

I find that making goals for myself helps me stay focused on what is important. If you don’t have a plan about what you want to accomplish, then you don’t really go anywhere.

One goal I have had for a few years was to publish my first app to the App Store. Every time I think I have some time to work on this goal, life gets in the way. I got an amazing job a few years ago that was stimulating but mentally exhausting, leaving me no mental energy to work on anything else. I took another job that gave me a nervous breakdown. I went through a divorce. I spent a year writing a book.

Each year the amount of complexity and polish required to publish an app increases. I always feel like everything is six months out of reach. By the time I finish an app, it will require another six months of work to be viable. Part of this was because I was a beginner. When everything is new, it takes longer. Once you understand the fundamentals and they become intuitive, then development speed increases dramatically.

While I was finishing up the book, I tried to do some research into game development because I wanted to hit the ground running when it was over. I wanted to get the long hard work of just learning the patterns and frameworks out of the way so that when I did sit down to write a game it would be easier.

I want to publish a game this year. I am tired of putting this goal off indefinitely because I keep getting offers to do other things. I don’t regret doing those other things, but I worry if I don’t make this a priority I will never do it.

Along with my goal to publish a game is to learn both Unity and Unreal. I know that these engines take a while to master, but I have seen many examples of people becoming proficient in them in as little as a month. I would like to have a better understanding of where people who use these engines are coming from. One impediment I had in learning graphics was a lack of familiarity with concepts like texture mapping and object imports. I think that having a better understanding of engines will be quite useful moving forward.

Another goal I have for 2018 is to do more work with graphics. I spent a year working on a graphics book, but I don’t feel I got to delve into it as deeply as I wanted to. Graphics is a complex topic. I feel like I finally established a foundation of my understanding that I can build off of. I worry if I don’t practice my new found knowledge that I will lose and forget it.

I’m very grateful to have a job that affords me enough time to pursue my passion projects while also being able to cover my bills. I am also incredibly grateful for my supportive boyfriend who wants to see me succeed with my special interests. Having been in a relationship with someone who actively discouraged my interests and interfered with my work, I know the value of having a supportive partner in crime. After a few years of instability, I am hoping that 2018 proves to be relatively quiet.