I’ve been working on a ChoiceScript project lately, and for the benefit of other ChoiceScript users, I thought I’d describe the debugging and testing techniques that I’m using.
If you’re not interested in using ChoiceScript, this is not the post for you, because it’s all technical from here.
This post assumes that you have ChoiceScript installed (available here) and that you have a work in progress.
Sections
Running Quicktest
Running Randomtest
Stat Blocks with Randomtest
Character Templates
Automatic Character Templates
Running Quicktest
What I want to know:
Is my game going to crash?
How to find out:
- Open quicktest.html (it’s in the root of the ChoiceScript folder).
As explained in the ChoiceScript automatic testing guide, quicktest.html will traverse every branch coded into the game, regardless of whether that branch is player-accessible or not.
Quicktest is like running a compiler. It will determine if the game will out-and-out crash, but it will not find any other errors. Read the automatic testing guide for more about how this tool works.
Running Randomtest
What I want to know:
Are any sections of my game inaccessible to the player?
How to find out:
- Open randomtest.html (also in the root of the ChoiceScript folder). By default, this runs 10000 iterations, which is a great number for most purposes.
- Check the box that says “Show line coverage statistics after test”.
- Click Start Randomtest.
This works by running 10,000 simulated players through your game. Every time there’s a choice in your game, each simulated player will make a decision at random. This will not tell you the true preponderance of player actions, because players don’t make their decisions at random (I hope!). It will, however, tell you what is possible.
After the test completes, it will print the start of every line of ChoiceScript code in your story, along with a scene name and a number. Copy that text into a word processor or text editor – I prefer Notepad++, which is my ChoiceScript editor of choice.
Search the text for the following:
[scenename] 0:
…where scenename is the name of your scene. (For example, if your scene is named chapter1, search for “chapter1 0:”.)
The number indicates how many of those 10,000 random playthroughs reached that line of text. In the case of a 0, that would be none, which is bad.
If you find 0 on a blank line, it probably just means that you jumped over it with a *goto. But if there’s any actual text there, it strongly suggests a code error. Check any “if” statement immediately in front of the line and verify that you haven’t typoed a variable name or entered an impossible value.
When I ran this test on my story just now, I found several paragraphs that weren’t accessible. Tracing the problem back, I determined that it was because I set a variable to “true” early in my story. This meant that the check
*if (variable = true)
was false, because true isn’t the same as “true”.
This is exactly the kind of error that randomtest will catch.
Stat Blocks with Randomtest
What I want to know:
What are the possible maximums and minimums for the stats in my game?
How to find out:
I could avoid this by conforming exactly to a detailed outline that contains the exact stat gains and losses possible in each scene. Or I could comb through each scene at the end and do the math myself. But this way is easier for me, especially because I’m using Fairmath with percentage stats, which makes the addition tricky there.
I have a scene called debug_stats_review.txt. It contains nothing but a list of stats, along with possible ranges that I want to examine, and a printout for each one.
Here’s an example of a readout for a skill stat.
*comment Skills – Tactics
*if (Tactics = 0)
Tactics 0
*if (Tactics = 1)
Tactics 1
*if (Tactics = 2)
Tactics 2
*if (Tactics = 3)
Tactics 3
*if (Tactics = 4)
Tactics 4
*if (Tactics = 5)
Tactics 5
*if (Tactics = 6)
Tactics 6
*if (Tactics = 7)
Tactics 7
*if (Tactics = 8)
Tactics 8
*if (Tactics = 9)
Tactics 9
It can go up past 9, of course, but I don’t need anything above 9 yet.
Here’s an example of a readout for a percentage stat.
*comment Reputation – Obedience (vs. Independence)
*if (Obedience <= 5)
Obedience between 0 and 5
*if ((Obedience > 5) and (Obedience <= 10))
Obedience between 5 and 10
*if ((Obedience > 10) and (Obedience <= 15))
Obedience between 10 and 15
*if ((Obedience > 15) and (Obedience <= 20))
Obedience between 15 and 20
*if ((Obedience > 20) and (Obedience <= 25))
Obedience between 20 and 25
*if ((Obedience > 25) and (Obedience <= 30))
Obedience between 25 and 30
*if ((Obedience > 30) and (Obedience <= 35))
Obedience between 30 and 35
*if ((Obedience > 35) and (Obedience <= 40))
Obedience between 35 and 40
*if ((Obedience > 40) and (Obedience <= 45))
Obedience between 40 and 45
*if ((Obedience > 45) and (Obedience <= 50))
Obedience between 45 and 50
*if ((Obedience > 50) and (Obedience <= 55))
Obedience between 50 and 55
*if ((Obedience > 55) and (Obedience <= 60))
Obedience between 55 and 60
*if ((Obedience > 60) and (Obedience <= 65))
Obedience between 60 and 65
*if ((Obedience > 65) and (Obedience <= 70))
Obedience between 65 and 70
*if ((Obedience > 70) and (Obedience <= 75))
Obedience between 70 and 75
*if ((Obedience > 75) and (Obedience <= 80))
Obedience between 75 and 80
*if ((Obedience > 80) and (Obedience <= 85))
Obedience between 80 and 85
*if ((Obedience > 85) and (Obedience <= 90))
Obedience between 85 and 90
*if ((Obedience > 90) and (Obedience <= 95))
Obedience between 90 and 95
*if ((Obedience > 95) and (Obedience <= 100))
Obedience between 95 and 100
Similarly, I could read this in finer increments than every 5, but right now every 5 suits my needs.
(Feel free to copy and paste the code above! If you just swap out the stat names, you’re good to go.)
In my scene_list in startup.txt, I have debug_stats_review listed as my final chapter.
*scene_list
startup
chapter1
chapter2
debug_stats_review
If I play through the game normally, this will just produce a choicescript_stats style readout at the very end. But when I combine it with Randomtest, I get a readout like this:
debug_stats_review 10000: *comment Skills – Tactics
debug_stats_review 10000: *if (Tactics = 0)
debug_stats_review 1366: Tactics 0
debug_stats_review 10000: *if (Tactics = 1)
debug_stats_review 3442: Tactics 1
debug_stats_review 10000: *if (Tactics = 2)
debug_stats_review 3338: Tactics 2
debug_stats_review 10000: *if (Tactics = 3)
debug_stats_review 1487: Tactics 3
debug_stats_review 10000: *if (Tactics = 4)
debug_stats_review 333: Tactics 4
debug_stats_review 10000: *if (Tactics = 5)
debug_stats_review 34: Tactics 5
debug_stats_review 10000: *if (Tactics = 6)
debug_stats_review 0: Tactics 6
debug_stats_review 10000: *if (Tactics = 7)
debug_stats_review 0: Tactics 7
debug_stats_review 10000: *if (Tactics = 8)
debug_stats_review 0: Tactics 8
debug_stats_review 10000: *if (Tactics = 9)
debug_stats_review 0: Tactics 9debug_stats_review 10000: *comment Reputation – Obedience (vs. Independence)
debug_stats_review 10000: *if (Obedience <= 5)
debug_stats_review 0: Obedience between 0 and 5
debug_stats_review 10000: *if ((Obedience > 5) and (Obedience <= 10))
debug_stats_review 0: Obedience between 5 and 10
debug_stats_review 10000: *if ((Obedience > 10) and (Obedience <= 15))
debug_stats_review 0: Obedience between 10 and 15
debug_stats_review 10000: *if ((Obedience > 15) and (Obedience <= 20))
debug_stats_review 0: Obedience between 15 and 20
debug_stats_review 10000: *if ((Obedience > 20) and (Obedience <= 25))
debug_stats_review 0: Obedience between 20 and 25
debug_stats_review 10000: *if ((Obedience > 25) and (Obedience <= 30))
debug_stats_review 1: Obedience between 25 and 30
debug_stats_review 10000: *if ((Obedience > 30) and (Obedience <= 35))
debug_stats_review 76: Obedience between 30 and 35
debug_stats_review 10000: *if ((Obedience > 35) and (Obedience <= 40))
debug_stats_review 741: Obedience between 35 and 40
debug_stats_review 10000: *if ((Obedience > 40) and (Obedience <= 45))
debug_stats_review 1882: Obedience between 40 and 45
debug_stats_review 10000: *if ((Obedience > 45) and (Obedience <= 50))
debug_stats_review 2543: Obedience between 45 and 50
debug_stats_review 10000: *if ((Obedience > 50) and (Obedience <= 55))
debug_stats_review 2567: Obedience between 50 and 55
debug_stats_review 10000: *if ((Obedience > 55) and (Obedience <= 60))
debug_stats_review 1436: Obedience between 55 and 60
debug_stats_review 10000: *if ((Obedience > 60) and (Obedience <= 65))
debug_stats_review 648: Obedience between 60 and 65
debug_stats_review 10000: *if ((Obedience > 65) and (Obedience <= 70))
debug_stats_review 105: Obedience between 65 and 70
debug_stats_review 10000: *if ((Obedience > 70) and (Obedience <= 75))
debug_stats_review 1: Obedience between 70 and 75
debug_stats_review 10000: *if ((Obedience > 75) and (Obedience <= 80))
debug_stats_review 0: Obedience between 75 and 80
debug_stats_review 10000: *if ((Obedience > 80) and (Obedience <= 85))
debug_stats_review 0: Obedience between 80 and 85
debug_stats_review 10000: *if ((Obedience > 85) and (Obedience <= 90))
debug_stats_review 0: Obedience between 85 and 90
debug_stats_review 10000: *if ((Obedience > 90) and (Obedience <= 95))
debug_stats_review 0: Obedience between 90 and 95
debug_stats_review 10000: *if ((Obedience > 95) and (Obedience <= 100))
debug_stats_review 0: Obedience between 95 and 100
This gets clearer if I pull out all the lines that are 10000 (because they’re *if statements) or 0 (because none of the randomtests reached them). Then I can see the actual range.
debug_stats_review 1366: Tactics 0
debug_stats_review 3442: Tactics 1
debug_stats_review 3338: Tactics 2
debug_stats_review 1487: Tactics 3
debug_stats_review 333: Tactics 4
debug_stats_review 34: Tactics 5debug_stats_review 1: Obedience between 25 and 30
debug_stats_review 76: Obedience between 30 and 35
debug_stats_review 741: Obedience between 35 and 40
debug_stats_review 1882: Obedience between 40 and 45
debug_stats_review 2543: Obedience between 45 and 50
debug_stats_review 2567: Obedience between 50 and 55
debug_stats_review 1436: Obedience between 55 and 60
debug_stats_review 648: Obedience between 60 and 65
debug_stats_review 105: Obedience between 65 and 70
debug_stats_review 1: Obedience between 70 and 75
10,000 random plays is a lot – but the more choices you have in your game, the more random plays you need.
This example illustrates it well because 1 player had an Obedience below 25 and 1 player had an Obedience over 75. That’s not very many! If I’d used a different random seed, I might not have gotten those 1-in-10,000 chances, and I might think it was actually impossible to have Obedience in those ranges.
Character Templates
This technique is for testing a later chapter without retesting the earlier chapters every time.
I keep the *create for every important stat in startup_txt. A highly abridged version looks something like this:
*create name “Nameless”
*create last_name “Nameless”
*create ey “she”
*create Tactics 0
*create Obedience 50
If I copy/paste all of those stats into a new file, and then replace “create” with “set”, that gives me a template. I change the actual values to whatever I want, add a “loading” reminder to myself about the character archetype, and save it under a name like debug_template_male_tactical.
Loading template_male_tactical
*set name “Primjer”
*set last_name “Zamjena”
*set ey “he”
*set Tactics 5
*set Obedience 60
When I want to test just the new chapter against this template, I alter the scene_list so that it includes my debug template immediately in front of my current chapter, like so.
*scene_list
startup
debug_template_male_tactical
chapter4
This will load the template and move into chapter 4 without any need to replay the first three chapters. And by running randomtest with this specific setup, I can see what chapter 4 looks like for 10,000 random copies of the template character.
Automatic Character Templates
This technique is for generating character templates from regular playthroughs.
Character templates are great, but it’s important to make sure that the character I’m creating is viable within the code. Since I’m setting up the template by hand, I might make a mistake like giving a character a Tactics of 9 when the game actually maxes out at Tactics 7. (Well, probably not, because I use the stat blocks described above – but bear with me.)
What I really want is to have a realistic character template. And the best way to do that is to play through the game, and then have the game produce a template from my playthrough.
EDITED 5/2/2015: Dani Church has suggested a superior code solution. I’ve preserved my original post below, but I’ve tested her script and it’s a great improvement.
The new file is called debug_generate_template, and it looks like this:
Generating character template:
set name “${name}”
*line_break
set last_name “${last_name}”
*line_break
set ey “${ey}”
*line_break
set Tactics ${Tactics}
*line_break
set Obedience ${Obedience}
Add it into startup.txt:
*scene_list
startup
chapter1
chapter2
debug_generate_template
The output will look like this:
Generating character template:
set name “Primjer”
set last_name “Zamjena”
set ey “he”
set Tactics 3
set Obedience 50
Just add an asterisk in front of each line, and you can paste it straight into a new template!
Templates are also useful if you have anyone playing your game on a chapter-by-chapter basis. (Hi, Mom!) If you generate a playtester’s template at the end of each chapter, you can then build a playtester-specific version when the next chapter is ready to go, rather than asking your playtester to replay all the prior chapters.
(Note that this only works if you don’t make major changes to the chapters. Otherwise, your playtester will be missing context.)
Hope this helps!
Happy ChoiceScripting!