Well, I’m about 300ish hours into my MMO development (part time over the past year or so) and the other day I decided to finally do some performance testing… way too late to be doing this sort of thing btw.  I didn’t think too much of it before because the GG engines are touted for their networking layer, so I figured it should be able to handle the tiny packets that I was sending back and forth with no problem.

I started my testing with what I thought was a modest number of players for an MMO… 1000.  Not too high, not too low.  The server grinded to a halt! and suddenly I got that ‘Oh shit! What have I done?!’ feeling.  So just to get a better measurement, I dropped it to 500, then 100 and finally to 50 before things started to perform normally.  Ouch!  Now, I’m not trying to get EVE Online numbers, but 50?!?!  My stomach sank a little as I thought about all of the wasted work and all of the work I would have to do to rewrite the server.

How could something awarded for its networking ability come up so short?  Well, it wasn’t the networking causing the bottleneck at all.  What I failed to consider was not networking, but the processing cycles needed.  Apparently, TGB doesn’t make the greatest server for a synchronous MMO.  The inherently single-threaded processing and the speed of torque-script were drastically overestimated on my part.

I can probably performance tune the TGB server more and move some things into native C++, but in the end I decided it would be easier to rewrite the whole server in Java which is my native language   While most people still have a negative impression of Java performance-wise, 1.6 is quite fast and it can usually be further performance tuned.

Moral of the story: performance test early so that you can change technologies if necessary and not have to rewrite a ton of code.

I haven’t finished writing the server yet, but I will post an update once I do some early performance testing.

After this happened a few times I replaced the battery.  To my surprise, the problem persisted, and not only that, but I started realizing that it only happens when its cold!  I checked over all of the power connections and everything looked good, so I turned to the All Knowing… Google!

I searched a bit and found out the real problem.  Apparently there is what I consider a design flaw in the anti-theft system.  The ignition measures the resistance of the key and if it doesn’t match the resistance that it was calibrated to then it triggers the anti-theft system and you can’t even attempt to start the car for another 10 minutes… yikes!  In extreme cold (or heat), the resistance of the key/ignition changes and the security system thinks that you are a thief!  I took the car to Saturn and they told me that it would cost some hundreds of dollars and I refused!  This isn’t just any abnormal malfunction!  From reading the amount of people that has this problem on the internet this is a problem that GM should have fixed years and years ago… I guess that’s why GM is in trouble and Saturn has gone out of business.

The Solution?  For older models, I remember reading that there was a work-around that includes starting the car and modifying some wiring, but for a newer model like mine (2006 Ion), I was out of luck.

I tried various things until I found one that worked:  I started bringing a lighter in the car with me and warming up the key and ignition before I tried starting it the first time (see disclaimer below).  The other important part is to turn the key very slowly (such that there is an even amount of tension on it) until the car starts.

I’ve even gotten to the point of breathing heavily on the key (the same way you do when you warm your hands up on a cold winter day) and then turn the key very slowly (such that there is an even amount of tension on it) until the car starts…. this seems to do the trick.

Not the most elegant solution, but it will save you some money.

DISCLAIMER: I take no responsibility for you starting yourself, your car, or any of your stuff on fire!

Standard Techniques:

• Object Oriented Programming (OOP) – concepts such as encapsulation, cohesion, and coupling are key concepts to understand.  Modeling data structures after real world objects has been around since the 60s, and widely used since the 90s but you still find people that refuse to do it.
  • Tight Encapsulation – this is a way to separate the public, potentially user-facing interface from the internal implementation of an object.  This is achieved by using different access modifiers such as public, private, etc.  It is very common to find public variables in an object that get accessed by other objects… this is bad.  If you then need to add new variables or change the way an object works internally (and you will), it creates a ripple effect and you now need to update all of the code for each of the objects that access the changed object.  The objective with encapsulation is to develop a stable public API for a particular class such that you can change the internal workings of that class (private methods and variables) without having to update the public interface.  What this accomplishes is that now you don’t have to update any of the other objects that are calling into the changed object… this equates to lower maintenance.
  • High Cohesion – this basically means that all of the responsibilities (methods) of a particular class are very closely related to each other.  The more focused a class’s purpose is, the less it will need to be maintained and the more reusable it will be.  For instance, if you had a Computer class, it could have a turnOn method and a compute method, but it shouldn’t have a print method.  Instead you would have a Printer class that has a print method that the Computer object may call.
  • Loose Coupling – this means that one class should not know too much about another class, more specifically, class A should not know anything about the implementation of class B.  This is achieved through tight encapsulation and good API design.  Class A should only know what class B wants the public to know about it.  Think of each class as strangers to each other class… you only give access to things (public methods) that you want the stranger to have access to and you never show the stranger your privates   When classes are too ‘friendly’ with each other (exposing things that should be private) it causes the same kind of ripple effect mentioned above.
• Class/Variable/Method Naming – names that you use in your software should clearly reflect their purpose and for the love of god, please don’t abbreviate unless its a well known acronym.  ssPos could be anything from space ship position to seriously stupid piece of shit… ok, its probably not that, but you get the point. spaceShipPosition would be a more appropriate name.  You would know exactly what it was when you look at it a month/year from now.  Some argue that the long length of these variable names will slow down your coding process… it wont… especially with most IDEs having some form of code completion these days.  Class names starting with an uppercase letter and variables/methods starting with a lower case letter and both continuing with camelCase is quickly becoming the widely accepted standard (IMO), but even if you don’t follow this at least be consistent.  Don’t use ufoCount in one place and alien_count or BulletCount in another.
• Don’t Hard-Code – its very easy while programming to start ‘hard-coding’ values directly into your code.  For instance, you have an condition that looks like ‘if(userCount > 100)’ in 3 or 4 places.  When in reality it should be ‘if(userCount > MAX_USER_THRESHOLD)’ and setting a constant called MAX_USER_THRESHOLD = 100.  Now if you want to allow more users, you simply update the constant value (or even better, put this in a config file as mentioned below) instead of updating it everywhere and potentially missing one.  Your code shouldn’t contain any hard-coded values unless they are what I call ‘universal truths’.  For instance, ‘if(userCount > 0)’, there’s no need to replace the number 0 with a constant called ZERO_USERS. 0 is 0… its never going to change.  This includes array sizes too.  You should try not to  make your arrays a constant size unless the size is dynamically created.  Instead you should try to use objects that can expand as entries are added (in Java – ArrayList, etc).  If for some reason you need to use a fixed size array you can then create one by using the size of the list (in Java – ‘new String[arrayList.size()];’)
• Commenting – we all hate to do it, but you would be surprised at how foreign your code looks a year from now (even less time if you don’t follow some of the stuff mentioned above).  I’m guilty of leaving my code comment-free from time to time, but I try to comment anything thats not immediately obvious to me.  Also, make sure your comments clearly describe what is happening, not just ‘//do stuff’.
• Refactoring – we all have deadlines and time crunches to worry about so there are going to be times when you need to write a quick and dirty piece of code… it works, but it irks you to your core (the good programmers out there know what I’m talking about).  When you write code like this, mark it somehow (for instance, Eclipse/Java uses FIXME comments) and this will be an indicator that you need to come back to it when you have more time.  When you do come back to it, write it in a nice low maintenance way and test it to make sure it does the same thing that the old code did.  The more of these little bits of ugly code you leave in, the harder to maintain the software becomes!

Advanced Techniques:

• Configuration File Processing – there are times when you have a ton of small little pieces of code that do pretty much the same thing except for small differences here and there.  Extract out the commonalities and write one piece of code that does the common parts for everything, then extract out the differences and put them in some form of configuration file (I prefer XML).  Now when the piece of code that is common runs, it can read from the configuration file to act on any differences.  An easy example is if you often have to change a hard-coded start/end date in your code in 4 different places, put the start/end dates in a config file and add code in those four places to read from the config file (which can be cached for speed).  This cuts down the number of places from 4 to 1 and removes the need to recompile.  The down side to using config files is that most of the bugs you find will end up being typos in the config file.  Also, don’t go crazy and extract everything otherwise you will end up writing your own scripting language.
• Reflection – this is a fairly new concept to most programmers, but this allows objects/variables/methods to be used/called just by knowing the name/signature of it at run-time (as opposed to design-time when you would normally do this).  This combined with the Configuration File Processing above could cut down on code drastically and virtually eliminate needing to update/compile your code.  For instance, you write a game that has user definable player characters (different look, different actions, etc).  Using the above mentioned techniques you could let the user create a config file that describes the player.  This config file could have one entry for a jpeg file for the character’s look and another entry that has a class/method name for the character’s action.  Now using reflection you could actually create the class that the user wrote and call their method… Nice!  Most newer languages support reflection and even a lot of scripting languages used in game engines support it (but probably don’t call it reflection).  Its a powerful thing to have in your mental toolbox.

Hopefully the above tips have shifted your thought process in the right direction and I won’t have to look at code like this from you:

// time it
if(nElpsdTm > 16400) {    <--- really hate the brackets here btw 
      Tmp = nElpsdTm;
      nElpsdTm = 500;
      sndMsgWTm(nElpsdTm);
}

If you have any other tips that you think would be useful to other readers, drop me a comment.

Honestly I can’t say for sure, but my current project is an MMO(RPG) and I’m using TGB to develop it.  Anyone who has used TGB might be thinking, “That’s suicide!”, but I think its possible and sooner or later (at the rate I develop games, definitely later) I’ll find out for sure!

I know TGB can be used to make quality games, so what more is there to consider in making an MMO?  Besides the sheer size of the project, the biggest thing that comes to mind is networking/data architecture to support thousands and thousands of players.

With that, here are some of the things I’ve already considered:

• Will TGB’s “turn-based networking” be an obstacle? The networking that comes with TGB was initially referred to as turn-based networking which to me implies that it is kind of slow/laggy, but in reality its more of an RPC-based networking so as long as you don’t need real-time ghosting for something like a twitch-based game, I think TGB may be able to hold its own.
• Can TGB handle that many concurrent connections? I asked this question a few different ways on the GG forums and the big shots tell me that the only realistic limit is the bandwidth and server hardware.  I guess if they are wrong then by the time I find out for myself it would be way too late to change to a different engine and you’ll surely find me hacking away at TGB source code to try and make it work.
• Does TGB have DB support? No, but there are some community source code mods that make it pretty painless to add SQLite support and it took me all of a couple hours to get this working. I hate making source code mods to TGB, but this one was definitely worth it… you simply can’t make an MMO without DB support.
• How much bandwidth will the server need? This is one thing that worries me because its not something that I can easily test.  As far as bandwidth goes, EQLive published EverQuest’s usage years ago as being about 1 KB/s per user download and .25 KB/s upload.  While I’m sure today’s MMOs (WoW, etc) use way more than that, I’m also estimating that my MMO will use quite a bit less than that as it doesn’t require as much information to be sent as often (in theory).  TGB also has a way to ‘tag’ strings between the server and each client such that subsequent sending of the same string gets sent as a tag ID rather than continuing to resend the same string over and over again.  This should cut down on the bandwidth usage.
• How will I scale up the server as more players join? Most likely I will host the TGB server myself until the beta is underway and then scale up to a VPS or Dedicated server as needed.  The server will need to be Windows based (mostly because that is my development platform).  I also need to be able to scale up quickly in case I get an honorable mention on one of the big gaming news sites, but I also don’t want to break the bank while I’m anxiously waiting for that to happen.

All-in-all I think I’m going to have my hands full for a long time, but I wouldn’t even try this if I didn’t think it was possible.

If you’re considering using TGB or another GG engine to make an MMO, check out this awesome list of discussions on the subject:
http://www.garagegames.com/community/forums/viewthread/84535

Even if you’re not going with TGB, here is a huge list of things to consider when making an MMO:
http://www.devmaster.net/forums/showthread.php?t=11656

I’m normally very logical and clear minded, but I’ve been very irritable lately (just ask my wife).  On top that, the clarity that I once had seems gone at the moment.  Turning 30 has motivated me to start thinking about what I want to be doing for the next 5 to 10 years and therein lies the confusion…

I have a great job as a software developer, but is that what I want to be doing for the rest of my life?  It wouldn’t be the worst thing that could happen… I also don’t think quitting my job and scraping pennies as a wannabe game developer is the best choice either.

A little clarity would go a long way right now, but anyway here are some other things I noticed that suck about turning 30:

• I can’t eat a pile of really spicy wings anymore… well… I can… and do, but its not worth it the next day (if you know what I mean)
• My hair is thinning… not that noticeable to other people, but I can’t help but notice the glimmer of my scalp through my hair when the light hits it a certain way.  I’ve been turning gray since I was 18, but that never bothered me as much as this.
• I feel like I have no time to do anything outside of work and sleep, and I don’t even have kids yet!
• Its easier to get hurt and slower to heal.
• I didn’t make my first million yet… oh, like your goals for turning 30 are so much better!
• My memory has been going downhill quick… I’ve got a ton of iphone reminders and post-its everywhere for things I’m supposed to do.
• There’s only a few years left until the end of the world.

That’s all I got for now… leave some comments and let me know what you think sucks about turning 30!

So, I haven’t done the newsletter part or the up-sell part yet, but I’m just about done giving the site a face lift (some of the ToW pages still need to be revamped).  So take a look around the Bantam City Games site and let me know what you think.  For those of you too lazy to click the link and browse around, here’s a screen shot of the main page.

With that being said, the project wasn’t a total waste.  Here are my observations:

The Good:

• I was able to include mouse-controlled game play, splash screens, menu, credits (all with screen transitions and some with final artwork), in 24 man hours.  That’s a huge accomplishment and quite frankly a credit to Torque Game Builder.
• I think a Christmas-themed game was a good choice from a sell-ability standpoint.  My thought here was that it should be easier sell a ‘less complex’ game when it is properly themed.  I don’t think anyone buys a Christmas game expecting to play it for months and months.
• I found a really cool site for generic vector graphics that can be reused in games (I found out that not all royalty-free sites have this type of licensing option).  www.VectorStock.com

The Bad:

• 24 hours is obviously too short a time to finish a game.  I half knew this going in, but ‘The 96 Hour Game Experiment’ just doesn’t have the same ring to it
• I started the experiment way too late in the year to get a Christmas-themed game to market (especially with me being extremely part time).
• I underestimated the amount of time it takes to test the game’s fun factor and throw away parts that aren’t fun and come up with/test new ideas.  This is where I spent the final hours.

The Ugly:

• Another unfinished project gets put on the shelf (at least until next year)

Overall, I’m glad I did the experiment and I hope to do another one at some point next year (with more realistic expectations).  This project made me realize how much I really can get done in such a short amount of time and that maybe being a part-timer isn’t the best way to go.  I’ll be putting some serious thought into how I can devote more of my time to game development in the near future.  For now, its back to working on my MMO…

I hope you enjoyed following along with the experiment and decide to stay tuned to see what’s next!

Anyway… I added some production quality art into the game (but still need to redo Santa and the houses) and finished up the Main Menu with some flashy artwork and here’s what I have so far:

I have most of the game play worked out, but the game seems too simplistic right now.  It feels too much like one of those flash games that you play for a few minutes at lunch… so I guess that means I need to enhance the game play a bit.

Most of the artwork is still placeholder stuff, and there’s no audio at all, so I need to get moving on that.  I think I’ll concentrate on the in-game art next.

I added an end-of-level stats screen which turned out pretty nice:

That’s my cue to pack it in for the night and hopefully I’ll be ready to go for tomorrow and/or Sunday.

Not too much to show for myself this round, but I did fix a bunch of issues with the game play mechanics and here’s a nice screen shot of the credits: