Basic Knowledge of Game Testing

To be a game tester, you need to be an avid game player. But playing games is not enough, you need to have a flair for finding problems. A computer game is further complex than a usual software application. Mostly games are made up of collection of essential components like rendering engine, database, animations, sounds, special effects etc.


The process and techniques that are normally used for software testing will also be functional for Game Testing e.g. Code Inspection, Path and Flow testing, Incremental Focus Testing etc.

You have to examine each and every part of the game. For example
1. The tester must search and explore the game as much as possible.
2. Menus in the game and their functions.
3. Sounds and the sound effects in the game e.g. music, lip sync and sound in conjunction with animation sequence. 
4. Logic of the game and game flow.
5. Objects in the game - terrain, world, texture etc.
6. Frame rate of the animation etc.
7. Scoring in the game.
8. Camera view, Zoom in, Zoom out, replay etc.
9. Attributes of a player, attributes of an action.
10. Different conditions to advance/move to the next level.
11. Any titles or text or fonts in the game scenes, spelling mistakes, legal text etc.
12. Game pause and different options related to it, hints in the game, any easter eggs.
13. Scrolling in the game.
14. Any movie clips in the game etc.
15. Different levels - with increased levels of difficulty.
16. Artificial Intelligence logic (if any).
17. Player movement, positioning.
18. Ease of use.
19. Any special effects.
20. Any other objects in the game - each object separately and how one object interacts with other.
21. Stopping and closing a game at certain level and opening the game at the same place you left it.
22. Test for platform compatibility (if any).
23. Loading time for the game under different scenarios.
24. Test for localization.
25. Networking should be tested thoroughly.

"Black Box" focuses on the functional characteristic of the game e.g. testing the UI – graphics, animations, overall look and feel, and the actual playing part.

"White Box" focuses on the architecture and integration characteristic of the game e.g. the use of a database, tests performed by developers prior to submitting new code for integration with the rest of the game, and so on.

Along with the positive scenarios, negative scenarios in game testing should not be ignored e.g. loading a game with less than required memory etc.

Things to keep in mind while testing games:
- You MUST be familiar with the game rules and then try to test the game against these rules.
- You should always try to check that the game should not crash or freeze in case of minor pauses or the characters should not move out of the bounds at any stage.

A table comparing a Gamer and a Tester: 


Gamer
Tester
Purpose
Fun
Mastery
Exploration
No
Yes
Genres
Limited
No Prefrences
Focus
Play Speed/accuracy
Skill/achievements
Avoids
Bad Games
Nothing

[Source of above table:Book: Game Development Essentials: Game QA & Testing by Luis Levy & Jeannie Novak]

You can prepare different DOCUMENTS on game testing, if the documentation will be good, the testing will be smooth and better: 
-Requirements document containing features, internal & external design of the game, how to do game testing, Definition of Done etc.
-Game Testing Test Plan (What kind of testing needs to be done, positive, negative etc).
-Test Design document (e.g. definition, guidelines, templates etc of preparing test plan document).
-Test cases depicting each and every scenario etc.

A realtime game tester requirement on a company website:
Your primary responsibilities as a tester will include bug testing and reporting, game balance and game feature testing. Game testing is fun, but it can be demanding. You'll perform traditional feature testing using testing methodologies that you'll learn on the job, and you'll assist the design team in identifying imbalances in game play. You will be an expert on the game and be able to notice any deviation from the correct design and functionality, no matter how small. You'll be expected to work hard, often focusing on very specific, detailed problems. Excellent written and spoken communication skills are crucial to relay what you found to the rest of the team. Patience and tenacity are important traits to demonstrate, as the job requires you to "isolate" every bug and figure out how to replicate it so that it can be fixed.

Some pretty useful resources on game testing:
1. A whitepaper on Systematic Approach to Game Testing

2. Game Testing on Windows, Mac & Linux Quality Assurance Testing in Game Testing Domain– A Case Study

3. Book: Game Testing All in One by Charles P. Schultz, Robert Bryant and Tim Langdell 

4. Testing Strategy for Games

5. Book Extract: Game QA & Testing - Ready, Set, Go!
Game Career Guide here presents an extract from Luis Levy & Jeannie Novak's book Game QA & Testing, a guide to entering the industry through the "traditional" route -- becoming a game tester. This helpful chapter provides information on how to really get in there and get the job.

6. Testing Video Games Can't Possibly Be Harder Than an Afternoon With Xbox, Right?

7. How Video Game Testers Work
Introduction to How Video Game Testers Work, The Need for Video Game Testers, Video Game Tester Responsibilities, Is Video Game Testing a Dream Job?, Becoming a Video Game Tester 

8. MEMOIRS OF A GAME TESTER
A personal story and tips on how to break into the games industry

9. Play games, give opinions, get free software!
Come to Microsoft, play games, give your opinion – and we’ll give you free software.

10. Book: Game Development Essentials: Game QA & Testing by Luis Levy & Jeannie Novak

11. Book: How to Become a Game Tester L. P. Klages

12. Book: Game Usability: Advancing the Player Experience By Katherine Isbister, Noah Schaffer

4 ways to get & count objects in QTP

Imagine simple and practical QTP tasks:
  • How to count all links on Web page?
  • How to get them and click each link?
  • How to get all WebEdits and check their values?

I'm going to show 4 approaches how to get lists of UI controls and process them (for example get their count).
As an example, I will work with links on Google Labs page. My goal is to get the list of links and count them.

I've added Google Labs page to my Object Repository and now it looks like:


I use Object Repository (OR) to simplify my demo-scripts.
Since the browser & the page were added to OR, we can use them later like:
Browser("Google Labs").Page("Google Labs").

Now we are ready to start!

  1. QTP Descriptive Programming (QTP DP) and ChildObjects QTP function
    The approach uses Description object, which contains a 'mask'
     for objects we would like to get.
    QTP script is:
    Set oDesc = Description.Create()
    oDesc("micclass").Value = "Link"
    Set 
    Links = Browser("Google Labs").Page("Google Labs").ChildObjects(oDesc)
    Msgbox "Total links: " & Links.Count
    The result of this QTP script is:

ChildObjects returns the collection of child objects matched the description ("micclass" is "Link") and  contained within the object (Page("Google Labs")).

2. Object QTP property and objects collections
    QTP can work with DOM:
Set Links = Browser("Google Labs").Page("Google Labs").Object.Links
Msgbox "Total links: " & Links.Length
I use Object property of Page object. It represents the HTML document in a given browser window.
This document contains different collections - formsframesimageslinks, etc.
And we use Length property to get the number of items in a collection.

The result is the same as for the previous QTP script:


    3. Object QTP property and GetElementsByTagName method
    Again, we can get access to 
    the HTML document and use its GetElementsByTagName method
    As the name says, 
    GetElementsByTagName method returns a collection of objects with the specified tag.
    Since we are going to get all link, we should use "a" tag.

    QTP script is:

    Set Links = Browser("Google Labs").Page("Google Labs").Object.GetElementsByTagName("a")
    Msgbox "Total links: " & Links.Length
    The result is the following:

    Note: There is another way how to select objects by tag name:

    Set Links = Browser("Google Labs").Page("Google Labs").Object.all.tags("a")
    Msgbox "Total links: " & Links.Length
    The result will be the same. 69 link will be found.

    4. XPath queries in QTP

    1. The idea of this approach is to use XPath queries on a source code of Web page.
      For example, "//a" XPath query returns all "a" nodes (= links) from XML file.

      There is one problem. Web page contains HTML code, which looks like XML code but actually it is not.
      For example:
      • HTML code can contain unclosed img or br tags, XML code cannot.
      • HTML code is a case-insensitive markup language, XML is a case-sensitive markup language, etc
        More details here.

      So, we have to convert HTML source code into XML. The converted code is named as XHTML.

      You can convert HTML documents into XHTML using an Open Source HTML Tidy utility.
      You can find more info about how to convert HTML code into XHTML code here.

      I will use the final QTP script from this page, a bit modified:

      ' to get an HTML source code of Web page
      HtmlCode = Browser("Google Labs").Page("Google Labs").Object.documentElement.outerHtml

      ' save HTML code to a local file
      Set fso = CreateObject("Scripting.FileSystemObject")
      Set f = fso.CreateTextFile("C:\HtmlCode.html"True, -1)
      f.Write(HtmlCode)
      f.Close()

      ' run tidy.exe to convert HTML to XHTML 
      Set oShell = CreateObject("Wscript.shell")
      oShell.Run "C:\tidy.exe --doctype omit -asxhtml -m -n C:\HtmlCode.html", 1, True ' waits for tidy.exe to be finished

      ' create MSXML parser
      Set objXML = CreateObject("MSXML2.DOMDocument.3.0")
      objXML.Async = False 
      objXML.Load("C:\HtmlCode.html")

      XPath = "//a" ' XPath query means to find all links
      Set Links = objXML.SelectNodes(XPath)
      Msgbox "Total links: " & Links.Length
      Note: you can download tidy.exe here for above QTP script.

      This QTP script leads to the same results - 69 links found:

    (Click the image to enlarge it)

    5. Bonus approah 
          Why don't you count all Wen page objects manually? :) Open a source code of the page and start  counting :)
    Just joking :)


    Summary:


    • I shown 4 practical approaches how to count Web page links.Similarly you can process images, webedits, etc
    • Each approach gets a list of objects.
    • First approach (QTP DP + ChildObjects) is the most easy
    • Second & third approaches (Object + collectionsObject + GetElementsByTagName) will work on Internet Explorer, because they use DOM methods
    • Fours approach is biggest but it is more powerful. It allows to use complex XPath queries.

    QTP - How to get all Object Indentification Properties?

    There are two ways how to get all properties of an object in QuickTest Professional:
    1. Manually
    2. Programmatically
    Use QTP Object Spy to get manually object properties.
    I've captured properties of ''Advanced Search" link from Google's page:



    So, QTP Object Spy shows these values:


    Using QTP Object Spy you can get Run-time Object Properties and Test Object Properties.
    It's possible to get these properties programatically:
    • The GetTOProperty and GetTOProperties methods enable you to retrieve a specific property value or all the properties and values that QuickTest uses to identify an object.
    • The GetROProperty returns the current value of the test object property from the object in the application.

    GetTOProperty differs from the GetROProperty method:
    • GetTOProperty returns the value from the test object's description.
      Test Object Properties are stored in the QTP Object Repository.
    • GetROProperty returns the current property value of the object in the application during the test run.
      QTP reads Run-time Object Properties from actual objects during the runnins can be read and accessed during the run session.

    That means that when you work with objects using QTP Descriptive Programming (DP), you will be able to access run-time object properties only (using GetROProperty function). Test object properties (using GetTOProperty function) will not be accessed, because QTP DP doesn't work Object Repository.

    There is a problem with Run-time object properties.
    In contrast to GetTOProperties (which returns the collection of all properties and values used to identify the test object), GetROPropertiesfunction does NOT exist!
    Once again - GetROProperties function does NOT exist!


    Well, how to get all Object Indentification Properties of an object? 
    Answer:
     We can read them from Windows Registry.

    The following registry key contains properties of a given test object:
    HKEY_LOCAL_MACHINE\SOFTWARE\Mercury Interactive\QuickTest Professional\MicTest\Test Objects\_Test_Object_\Properties

    For example, I've opened:
    HKEY_LOCAL_MACHINE\SOFTWARE\Mercury Interactive\QuickTest Professional\MicTest\Test Objects\Link\Properties 
    and I've got properties of Link object:


    Please note that you can find the same Link Identification Properties in QuickTest Professional Help:

    QTP Object Identification Properties can be used:
    • in the object repository description
    • in programmatic descriptions
    • in checkpoint and output value steps
    • and as argument values for the GetTOProperty and GetROProperty methods

    So we have to read all Identification Properties from the registry.
    This QTP code reads Link Identification Properties:
    Const HKEY_LOCAL_MACHINE = &H80000002
    Set oReg = GetObject("winmgmts:{impersonationLevel=impersonate}!\\.\root\default:StdRegProv")

    sKeyPath = "SOFTWARE\Mercury Interactive\QuickTest Professional\MicTest\Test Objects\Link\Properties"
    oReg.EnumValues HKEY_LOCAL_MACHINE, sKeyPath, arrNames

    As a result, arrNames array contains all names of properties.
    To prove my words I use this QTP script:
    sNames = "Identfication Properties:" & vbNewLine

    For 
    i = 0 to UBound(arrNames)
        sNames = sNames & arrNames(i) & vbNewLine
    Next

    MsgBox 
    sNames
    The result is:
    Compare these Link Identification Properties with properties from the registry. They are the same!

    So, we can read names of properties.

    Next step is to read their values. It can be archived using GetTOProperty or GetROProperty.
    Also, I'm going to show how GetTOProperty and GetROProperty work for Test Object (located in QTP Object Repository) and Run-time Object (actual object, created during the run session).
    1. Properties of Test Object
      QTP script is:
      ' Link("Advanced Search") is an object from QTP Object Repository
      Set TestLink = Browser("Google").Page("Google").Link("Advanced Search")

      sNamesTO = "GetTOProperty for Test Object" & vbNewLine "Identfication Properties: Values" & vbNewLine
      sNamesRO = "GetROProperty for Test Object" & vbNewLine "Identfication Properties: Values" & vbNewLine

      For i = 0 to UBound(arrNames)
          sNamesTO = sNamesTO & arrNames(i) & ": " & TestLink.GetTOProperty(arrNames(i)) & vbNewLine
          sNamesRO = sNamesRO & arrNames(i) & ": " & TestLink.GetROProperty(arrNames(i)) & vbNewLine
      Next

      MsgBox sNamesTO
      MsgBox sNamesRO
      • Test Object Properties of Test Object
        Test Object Properties of Test Object and their values are:

      • Run-time Object Properties of Test ObjectRun-time Object Properties of Test Object and their values are:

            2. Properties of Run-time Object             QTP script is:

    ' Link("text:=Advanced Search") is a dynamic run-time object
    Set TestLink = Browser("Google").Page("Google").Link("text:=Advanced Search")

    sNamesTO = "GetTOProperty for Run-time Object" & vbNewLine "Identfication Properties: Values" &vbNewLine
    sNamesRO = "GetROProperty for Run-time Object" & vbNewLine "Identfication Properties: Values" &vbNewLine

    For i = 0 to UBound(arrNames)
        sNamesTO = sNamesTO & arrNames(i) & ": " & TestLink.GetTOProperty(arrNames(i)) & vbNewLine
        sNamesRO = sNamesRO & arrNames(i) & ": " & TestLink.GetROProperty(arrNames(i)) & vbNewLine
    Next

    MsgBox sNamesTO
    MsgBox sNamesRO
    • Test Object Properties of Run-time ObjectTest Object Properties of Run-time Object and their values are:


    • Why almost all properties are empty?

      As I said, GetTOProperty function gets values from Test Object, which is stored in QTP Object Repository. Since Run-time Object is a dynamic object, it's not stored in QTP Object Repository. That's why GetTOProperty function cannot read object's properties.

      Look at the above screenshot again. The only one property ('text') contains its value ('Advanced Search'). We used this property to create description for our link:
      Set TestLink = Browser("Google").Page("Google").Link("text:=Advanced Search")
      That's why this Run-time Object contains the only property.
    • Run-time Object Properties of Run-time ObjectRun-time Object Properties of Run-time Object and their values are:

    As you can see, we got the same Run-time Object Properties both for Test Object and for Run-time Object. I can explain it.
    During the run session, QTP creates a Run-time copy of Test Object. That's why Run-time Object Properties were the same for Test Object and Run-time Object.

    Note: You can download final QTP script here.