Square Off Post Mortem

What worked Well

Use of XNA Development Platform

Using a cross platform framework eased the steep learning curve of developing console games. Having a PC build with xml scripting helped us quickly prototype ideas (especially for level design). The flexibility of c# was an absolute god-send. Reflection allowed us to configure every level and game object with xml, so even though the tools lagged behind, maximum tweakability was maintained. With a little help from other developers, new team members could easily pick up and run with the tools, making the content creation pipeline reasonably streamlined in a short space of time.

Publishing on Xbox Live Indie Games

Before XBLIG, it was very hard for an indie developer to get a game developed and released on a current-gen console. There were too many initial overheads that would create too much financial risk for part time game development. Being able to develop and test directly on the Xbox, release for playtest or review and then distribute directly to the XBLIG channel easily made this usually monumental task relatively easy.

Iterative Prototyping

Our main aim was to create and submit a game to the Dream Build Play 2009 competition. Although we succeeded, we hit a small disaster 6 weeks from submission deadline. The then current game we were working on, Throng was not working out design-wise. We then decided to go back to the proverbial drawing board and pretty much start again. With not much time left, we turned to iterative prototyping to quickly churn out game play concepts to see what could be done in 5 – 6 weeks. This is basically how Square Off came into being and after a few concepts, code tweaks and discussions, we got development going again. We started with basically no graphics besides simple pong-like shapes (squares for players and circles for enemies, sound familiar?) and once we found a good balance between easy to pickup controls, decent game-play and fun factor, we began developing the final art assets. Thankfully, most of the core architecture was complete and compatible with the new style of game we decided to go with in the months leading up to the change in game direction.

Small Core Team and Deadline Pressure

Having a small team was good when developing the game because decisions were made quickly and since we were only doing this part time, it forced us to keep things simple and achievable. One of the biggest factors of our success in finishing this project was having the Dream Build Play Deadline looming overhead. When we released the patch, since there was no deadline it dragged on for more than double the time it took to develop the actual original game!

Hosted Version Control (SVN)

Revision control system is very important. We found that seeing the change-set notes via email (which the SVN service is sending) and knowing what was going into each build that was tested was very valuable. We decided to use Codesion for our SVN hosting.

Issue Tracking (Flyspray)

Having a list of bugs and issues that everyone can see, comment on and resolved was extremely useful. It reminded us what needed to be done before we could release the game to the public, as well as document new ideas for the game and vote on the best ones to implement. This seemed to work well even though we had reservations at the start. The key to its success was that most developers were very active in the system, making comments and creating issues etc. This was really amazing to see, and made it worth the effort to select and setup a bug tracking system. We have some concerns about how it might be able to scale up… but we can cross that bridge when we come to it.

Internal Developer Mailing list (Google Groups)

We used an internal mailing list that everyone in the team was on to communicate and discuss items and issues, much like our own personal forum. This had the same positive effect as issue tracking, this worked well because people actually used it and it was a good record of ideas and conversations that would have been forgotten otherwise.

New Team Members

Getting new team members during the patch was a mixed blessing because a lot of new ideas came into play. Whilst this did complicate the process, it was far outweighed by the new skill sets available to the team and the quality of those new ideas. Fresh eyes for feedback and implementation of bug/version tracking proved invaluable. Both of the two new developers started out in a QA capacity, meaning the number of bugs that were found in the original game and update features was simply astounding. Although the bugs took a long time to fix and verify, the alternative is that they would be overlooked and eventually found by our paying customers.

What could be Improved

Marketing

No matter how much we did, this could always be improved. This is the bane of every indie developer. After you finish the game, there is a whole new range of issues to address; getting the word out about your game. Although we used various sites that help issue press releases and information for you, there is still a large amount of work involved, whether it is sending emails, creating a video trailer showcasing new features or even updating the website, there is enough work in this phase to warrant getting a person full time on this job. And it is a very important job, if people don’t hear about your game somehow, how are they meant to buy it?

Scope Creep

This turned into a big issue for us. We suffered from perfectionism. “Lets add this”, which follows with “this, then this,” and so on. I am glad we dropped new features like online and achievements as we would still be working on the patch for another 6 months. Next time we definitely have to set monthly goals and stick to them, even better a reasonably detailed game design document/bible so we minimise the scope creep problems that arise.

Not Enough Pre-Production

As mentioned previously, this was a problem as we drastically changed direction late in the development stage. We could solve this by creating a design document, by writing the ideas down in detail to see if the idea could work, and can save us a lot of wasted time later. Once we set what we are doing and where we need to go, then we can begin development. It worked out alright for us on a small project like Square Off with a small team, however with more people and a bigger project; the same approach would prove to be dangerous.

Being more prepared for testing sessions

During the testing phase of Square Off, we regularly got together to test various builds of the game for verification of bug fixes to finding new ones. Although this was useful, some points for us to consider for next project were:

Needed a stopwatch – this would be handy to compare things such as level/game completion time and various time based bugs we came across
Try a few different builds (and a known stable build) during the same session to see what changed between versions if new bugs are found
A brief testing plan, based on recent changes or areas to focus on
Book the session in advance
Bring a printed list of issues to play test/verify that cannot be verified by only 1 person

Testing was not varied enough

Play testing should have been conducted with different player groups (different ages, skill-sets, experience). We utilised friends, family and co-workers where we could, but we could have been more pro-active with this.

Website/SVN/Database/Flyspray backups

Our website was hacked, and needed to be restored. Luckily we had offline backups, however keeping backups of various tracking and working files is something we should keep up to date more often

Managing non-development tasks

It is hard to see how non-development tasks are progressing such as general business matters, future project ideas progress, reviewer / advertising follow ups, etc
Makes it hard for new members in the team to help out since it these tasks are not adequately tracked

Didn’t stick to deadlines

We didn’t really set dates for deadlines, or if we did, we slipped without much concern. If this was a full time business then this would be unacceptable
Developers always want to make the product better and there are always things to do to improve it, but we need to be more business minded
Possible solutions to this would be to set what deliverables are due at end of each month as well as cutting / deferring issues more ruthlessly

Our time is free… but

We have practically zero cost, but we also have lost opportunity cost. For example if we spend 12 months doing nothing, then we have forever lost the opportunity to develop a profitable product in that time.
While our time doesn’t cost the business anything currently, we need to consider how we can optimise the use of everyone’s time, to make sure people stay motivated and maximise profits from all released titles.
Time spent on the patch (~6 months) has cost us the lost opportunity to spend the time building a new profitable product.
What else could we have developed in this time?
How much might that have earned us?
What is the cost vs. benefit of doing the patch?
(The punt/experiment is that the patch release and PR will increase sales more than the potential sales from a new product developed in those 6 months.)

Projects are a balance between cost, schedule (time) and quality. If you assume that cost is zero, then it is just a balance between time and quality. We seemed to be thinking about time and quality, but not considering the cost.

Artists have had very little art to do in the patch

Work in parallel on different projects to ensure there is no artist downtime
Choose projects to work on that will make the most use of artists, while minimising software work
- Games that have many levels
- Games that have many characters/enemies
Don’t choose projects that only require software work (unless running in parallel)

Did not start on the next project early enough

We should have started on the next project much earlier
Make better use of team members that could prototype new concepts while testing was being done
Prototype on different systems to help with estimates and deciding on which project to work on
Gives the artists something to work on during the testing phase of the release

Thanks to Josh Stewart and Adam Matera for contributing the bulk of this post mortem. I think it will help us focus and prioritise to improve our subsequent projects.