7 posts tagged “testing”

I'm going to take a break from the Audio-Visual tutorials and talk a little bit about game development, specifically the teams that make it up. While game development can be a dawning task, I have come up with a few helpful hints on putting together a team that should help keep you on track.

1. Artists - Sketching
   You don't need much if you're just starting out, but it's always good to be able to sketch up a concept, if not at least have someone on the team who can. Sketching can be anywhere detailed as it needs to be. Concepts shouldn't take very long to sketch depending on it's scope. A concept artist should be able to take what the game designer(s) create(s) and sketch that into a visual. Often times a sketch artist will need to go into more detail by using actual rendering tools like Photoshop or something similar.
   
   
2. Artists - Modeling
   If you are going to be working with 3D games it is a necessity to have someone on the team with enough experience in 3d modeling. This artist should be able to take what the sketch artist has come up with into a realistic 3d visual. Unless there is a texture artist available, texturing will often be performed by this artist as well.
   
   
3. Artists - Texturing/Shading
   When resources are available, a game development team may also include texture artists. In this role, the person should be able to combine the efforts of the sketch artists with the modelers in order to create the final visual in the title. Texturing itself is usually done in layers depending on the amount of detail necessary. It is not uncommon for these artists to also perform shading techniques on top of their normal tasks.
   
   
4. Artists/Musicians - Sound
   Sound, because no game is complete without the necessary audio-visual aspects. This artist should be in charge of providing all the necessary sound effects, music and fonts needed for a given game title. This role may be played by a developer but it is usually filled by ones with musical and artistic skills.
   
   
5. Designers - Game Design
   For the most part game design is a collaborative effort, but it is best to have someone who concludes the final design on game concepts. The game designer should be able to explain what to include as well as exclude from a game title. Often times this designer will perform consulting on visual aspects of the gameplay as well as general environmental factors that make up the game.
   
   
6. Developers - Game Development
   The core of the team who's role is to put the pieces of the puzzle together. This role can vary in many scenarios depending on the requirements of the game. The job may include writing networking code, artificial intelligence, camera angling, character movement and interaction among other tasks that the game may require. Be sure to understand the necessities of the game and the availability of tools before choosing this role and the skills.

While it's not entirely necessary to have the above people on your team, you or the people on your team may fill one of the above roles at some point in your game development cycle. As you progress, it will be necessary to also have testers on the team, but I've always approached this issue with test-driven development.

Aside from roles on a game team, there are a few things to keep in mind about creating a game. The first is the concepts included or excluded in a game. When in doubt, prototype your concepts as quickly as possible. Quick designs or sketches for a prototype shouldn't take more than an hour or two and if it does then it needs to be simplified. Building on concepts isn't necessary bad though; the larger and more complex a concept gets, ideally, the more prototypes you should already have. By then end of your game designing phase you should be able to present each aspect of the game with prototypes.

That's all for now :)

As a quality assurance developer for NORMA (Neumont ORM Architect for Visual Studio) this quarter I get the priveledge of working in many, many... many different orm models for test cases. So I thought I'd share some of the humorous models that have come up over time.

Chuck Norris has no reference scheme.

 

T-Rex eats Person in Game

Alright, this time I'm going to get right to the gut of quality assurance and talk about the ideal kind of testing environment when it comes to software, big or small.

Firstly, what kind of project are you planning on building? Is it a website? A desktop application? A mobile phone application? Realistically, it can be any project so long as you follow these 12 guidelines.

1. Do you have source control?
       It's always good to have source control and especially for the type of test driven environment you're looking for it will be even better for running all of those tests.
2. Can you build your solution in 1 Step
       Most of us are using Visual Studio to build our projects but nevertheless you should be able to build your solution in 1 step whether it be building it for debug or compile and/or including deployment. It's a pretty common scenario where you want to deploy your solution immediately when you're done.
3. Do you perform daily builds?
       If you're working on a project that goes day in and day out of work you should ensure that you at least build once a day or at night to follow up tests and issues in the program.
4. Do you have a bug database?
       Bug databases are used for instant tracking of all of your bugs, whether automatically reported by the system or typed in by a user or developer on the team. It's very good to have one of these if you're working in a larger team. But even in a small team (like yourself) it can be pretty handy to organize everything into a simple database of bugs. Makes it easier to clean things off.
5. Do you fix bugs before writing new code?
       This one is tough if you're not working directly off test driven code. What test-driven code means is that for every feature or series of implementations you have written tests to ensure that all logic and implementations are valid or work according to a standard. Think of test-driven development as writing the model that you want your project to become and the code that you are writing to be tested against are the drafts to the sculpture.
6. Do you have an up-to-date schedule?
       What are you working on? What needs to be done? Who's working on what? Try not to think of schedules as some mundane constraint who's only use to keep the manager out of your hair. Schedules are incredibly valuable tools for time management. They teach you to manage your time wisely and keep you from working day in and day out on a silly problem that may mean nothing in the long run. In our industry we are too often thinking about the how and the what-if scenarios of software development but we rarely take a step back and ask whether or not we should do something rather than if it's possible.
7. Do you a specification?
       If you're working on something that other people are going to be either working on or integrating with, please please please for the sake of all time and humanity, write healthy comments and anything else that will help people understand how to work with said system. Tutorials, samples, anything that relates to your system is really helpful for others to use their time just as wisely and timely.
8. Does your team have quiet working environments?
       I know, you're thinking, "Wait a minute. What does quiet working environments have to do with software quality assurance." Well believe it or not, but a quiet working environment is basically a large white board to the mind of a developer and a tester. It ensures that not only can someone hear their own thoughts, but as a team it makes it easier to discuss and collaborate without disturbance.
9. Do you own or use the best tools money can buy?
       Better tools, newer technologies anything that can aid you and speed up your time in development or your goals is far more useful than ever, especially to your users who are so eager to see what you've come up with.
10. Do you have a group of people or person who's sole purpose is to write tests?
        This is more so for larger groups, in small groups you will be the tester and as a developer it's best practice to get in the habit of testing your own code before hand.
11. When you have new people on the project, do you have them write code before joining the team?
        Usually it's a great idea to have the person you're interviewing actually write code related to what you're working with or what you plan to have them working with. Sometimes it doesn't really have to be a test of knowledge but to allow the person to get a feel for the code that he/she will be working with and it allows you to adapt to whatever practices or standards he/she may already go by.
12. Do you perform hallway usability testing?
        Hallway usability tests are done by simply taking your laptop out into the hallway and showing what you've made. It's a great way to get suggestions and get other people excited about your creations. It's a great motivator if the people you test give back constructive criticism. And if you've followed the above guidelines this step should be no problem because you'll never run into a situation of a useless feature or a bug. You can explain it with confidence and passion!

What do I use for my test-driven quality assurance setup?

1. Subversion Web: I try to always have a subversion setup that I can access via the web. These are useful for the fact that most of them have wikis for documentation and logs that come from subversion. It makes it especially easier for deployment since I can just create a release from the website.
   I've been using: unfuddle lately. I think it's great!

Software is complex, in fact it can be infinitely complex depending on how the product is developed if extensibility is a factor. There are literally millions of scenarios that can occur not only the development of a product but even after a product has been deployed. Many times after a product is deployed (after getting customer feedback) developers will run into many issues that they did not foresee originally. When software is deployed and run on a client machine there are scenarios that may work for one user but not for another; like settings and runtime environments.

Because of these vast differences in environments, developers use common test utilities to cover their code and test against common scenarios that may break working code. Surely you should test to see if you code works, of course it should work you wrote it. There are, however, many scenarios that haven't been tested even after coverage is 100%. As such, after developing an application, there needs to be a standard for finding out possible failures in the code. I've noted some of the principles involved in testing basics for developing software.

• Find problems before customers do
• Find problems customers care about
• Ensure all features of the product are acceptable by the customer.
• Provide timely feedback on functionality of new features.
• Provide timely feedback on stability of old features.
• Role is not to prove the program works
• No way to fully test all possible program paths
• All software ships with some problems
• A problem is anything the customer doesn't like.

Some testing approaches have been known to include:

• White Box Testing (aka Glass Box Testing)
• Full access to the source code
• Static analysis
• Code reviews
• Static analysis tools (FxCop, etc)
• Dynamic Analysis
• Step through code in a debugger, examine program state
• Profiling (line coverage, branch coverage, performance)
• Black Box Testing
• No assumptions on implementation
• Two approaches
• Automate user interactions
• Use public API provided by the program
• Both techniques have value
• An UNDONE comment in code may reveal a subtle bug
• But customers don't read or profile code

Of course we can look into the types of tests here. Such as in Ad-Hoc tests, where the developers simply run the program and use it themselves. This makes it fairly easy and extremely quick to get coverage. Through ad-hoc tests we can get to major bugs very quickly. Next to Ad-Hoc is Manual tests, where developers write test cases to explicitly test the product while recording results and running them by hand. Opposed to both kinds is the Automated testing. In automated tests, a program is written to run the program automatically and test scenarios automatically. Generally automated test programs with the amount of code written can be much larger than the actual feature. In summation of the previous test types, is regression tests where developers run a test of test scenarios done in any other type of test.

Let's take a step back from testing and think about the psychology of testing itself.

1. Testing is inherently adversial
   
   • Programmers are trying to ship and you're telling them they can't.
   • Product managers are losing money in development costs and lost revenue and you're telling them they'll lose more if they ship now.
2. Never make it personal
   
   • Issues always outlast programmers, especially bad ones. When you write a test scenario, the code could be re-written and re-written over and over but the test scenario lives through all the code rewrites.
     
3. You have to carefully sell the things you really care about.
4. Clear expectations in bug reports
   
   • You can't simply put "X Doesn't Work." in a bug report. That doesn't explain anything to anyone reading this. A developer won't have your test code, you have to be able to explain enough to allow anyone to go right to where they need to be to fix code.

Aside from these basics, there are plenty of other scenarios that you will want to look into when in a proper test driven environment.

I've always loved some of the tools that are built for the developers ease like NCover, NUnit, TestDriven.Net, Krypton, Community Server, NDoc.. and much more. Brennan compiled and posted a list of Web Development Tools for the Power Developer. Jim Holmes who has co-authored a book, Windows Developer Power Tools, wrote a list of all the tools that make up our happy developer lives.

XSD.exe	Krypton Toolkit		Community Server
WSCF	ControlSpy		Subtext
Codus	Visual Studio Express Editions		Flexwiki
MyGeneration	SharpDevelop		XP Remote Assistance
Peli's Reflector Addins	MonoDevelop		Skype
SourceMonitor	SnippetCompiler		GAIM
CR_Metrics	Notepad2		TFS Admin Tool
Ndepend	Regulator		NHibernate

More on the list after the jump.

Dr. James McCaffrey has a very nice way to perform testing automation on Custom Ajax driven websites. The testing automation works really simply since everything is executed and logged via client-side javascript code. 

The popularity of Web applications that use AJAX (Asynchronous JavaScript And XML) technology has increased steadily over the past year. When written correctly, AJAX can yield significant improvements in performance and user experience compared with non-AJAX Web applications. However, because AJAX Web applications work asynchronously, traditional synchronous test automation techniques generally don't work. In this month's column, I present a technique that allows you to write lightweight test automation to verify the functionality of AJAX Web applications.
More after the jump.