Embedding Lua

Posted on by
Reply

Awhile ago I started messing with Lua and seeing how I could implement that into my engine. There’s a number of reasons why you would want to add scripting functionality, but for me I wanted to allow users to mod the API set, and to allow faster prototyping. Right now I have about 27 projects that need to be built if I was to do a rebuilt on the entire engine, and that could take 25-45 minutes depending on which computer I’m initiating the build on. Obviously scripting the prototype out first could save me a lot of time so I set out to add scripting to Core. After looking at the more popular scripting languages, I decided to pick Lua. Lua is a very common language in gaming, has great support for C/C++, in fact it was created to work with C and C++.

Before you read this understand, this is not a tutorial on how to use ToLua++, there are plenty of them out there. This is an overview of my process of using it, and how to automate some more of the tedious tasks required before the Lua wrapping code can be created.

I found a tool called ToLua++ that will read your formatted head files and automatically create C++ wrappers to those methods that will allow Lua to access them directly. This is a big time saver because all you need to do is create a simple batch that will run ToLua++ on some specified headers, import the outputted C++ code into your project, and your done (theoretically). It’s not always just punch and run, the ToLua++ tool doesn’t have a full and complete C++ parser, so there are some formatting steps you need to do, but I already coded that up in the “Asset Manager” tool I’ve been working on so I’ll run through some of the logical steps I’ve added.

ToLua++ and Manual

Check the manual on some of the manual filter you will need to do to your headers, but here is the logic I did to automatically format my own headers…

Here is the tool I made to help automate the entire process:

  • The top directory is the root of all my include files.
  • After I click on the “Directory List” it will populate all the header files in all child directories.
  • From that point I can choose which includes I want to have wrapped, and which header files I want not to include.
  • After I click “Generate”, all the ToLua++ pkg files are created and example commands are created so I can simply copy and paste the commands into the command prompt if I want to create the ToLua++ package.

What the generate tool does is it filters the header files I want to be wrapped for Lua, and filters and formats them so ToLua can better read them. Here is an example package file:

    class AbstractVehicle : public AbstractLocalSpace 
    {
    public:
         ~AbstractVehicle() { }
 
         float mass (void) const = 0;
         float setMass (float) = 0;
         float radius (void) const = 0;
         float setRadius (float) = 0;
         reVector3Df velocity (void) const = 0;
         float speed (void) const = 0;
         float setSpeed (float) = 0;
         reVector3Df predictFuturePosition (const float predictionTime) const = 0;
         float maxForce (void) const = 0;
         float setMaxForce (float) = 0;
         float maxSpeed (void) const = 0;
         float setMaxSpeed (float) = 0;
	void update(const float currentTime, const float elapsedTime) = 0;
    };

This is a class from the AI engine, so now I have this functionality available to me using Lua. It looks like any other C++ class declaration, and about 99% of is, with a few exceptions:

  • Pre compiler commands are removed.
  • typedef’s are removed.
  • Array’s with an element size defined using #define is removed.
  • friend declarations are removed.
  • using namesapace lines are removed.
  • Custom primitive types are replaces with the standard C/C++ names.
  • I’ve had problems with virtual in the past so now I just remove the word.
  • I remove all private and protected members.
  • Finally, ToLua doesn’t seem to play nice with some of my template declarations, so I’ve hardcoded the Asset Manager to find those lines and remove them.

Here is an over all flow of what my tool does when filtering the class declarations:

public void GeneratePackages(CheckedListBox clb, string outputDir, ListBox ingnoreList, TextBox example)
        {
            var pkgsCreated = new List<string>();
 
            var di = new DirectoryInfo(outputDir);
            if (di.Exists)
                di.Delete(true);
 
            di.Create();
            example.Text = string.Empty;
 
            foreach (string dir in clb.CheckedItems)
            {
                var itemsDir = new DirectoryInfo(dir);
                var tempFileName = outputDir + "\\" + itemsDir.Name + ".temp";
                pkgsCreated.Add(tempFileName);
                using (StreamWriter writer = new StreamWriter(tempFileName))
                {
                    example.Text += "tolua++ -n " + 
                        itemsDir.Name + 
                        " -o " + itemsDir.Name + 
                        "_lua.cpp -H " + itemsDir.Name + "_lua.h ./" + itemsDir.Name + " \r\n";
 
                    foreach (FileInfo file in itemsDir.GetFiles())
                    {
                        if (file.Name != "pkg" && !ingnoreList.Items.Contains(file.Name) && 
                            (file.Extension == ".h" || file.Extension == ".hpp"))
                        {
                            using (StreamReader reader = new StreamReader(file.FullName))
                            {
                                while (!reader.EndOfStream)
                                {
                                    string line = reader.ReadLine();
 
                                    // **************** Validate the line *************************
 
                                    // Many issues with this
                                    if (line.Trim().StartsWith("#"))
                                        line = string.Empty;
                                    // Not needed
                                    if (line.Trim().StartsWith("typedef") && !line.Contains("typedef struct"))
                                        line = string.Empty;
                                    // Variable definition
                                    if (line.Contains("NUM_TEAMS"))
                                        line = string.Empty;
                                    // Doesn't like friends
                                    if (line.Trim().StartsWith("friend"))
                                        line = string.Empty;
                                    // Don't need this
                                    if (line.Trim().StartsWith("using namespace"))
                                        line = string.Empty;
                                    // Has a problem with dereferencing
                                    if (line.Contains("** ") && !line.Contains("/*"))           
                                        line = string.Empty;
 
                                    // Core specfic
                                    line = line.Replace("CORE_EXPORT", "");
                                    line = line.Replace("NATURE_EXPORT", "");
                                    line = line.Replace("__declspec(dllexport)", "");
                                    line = line.Replace("virtual", "");
                                    line = line.Replace("f32", "float");
                                    line = line.Replace("s32", "int");
                                    line = line.Replace("u32", "unsigned int");
                                    line = line.Replace("c8", "char");
                                    line = line.Replace("s8", "signed char");
                                    line = line.Replace("u8", "unsigned char");
 
                                    // AI specific
                                    line = line.Replace("template <class Super>", "");
                                    line = line.Replace("template<>", "");
                                    line = line.Replace("template <class ContentType>", "");
                                    line = line.Replace("template< class PathAlike, class Mapping, class BaseDataExtractionPolicy = PointToPathAlikeBaseDataExtractionPolicy< PathAlike > >", "");
                                    line = line.Replace("template< class PathAlike, class Mapping >", "");
                                    line = line.Replace("template< class PathAlike, class Mapping, class BaseDataExtractionPolicy = DistanceToPathAlikeBaseDataExtractionPolicy< PathAlike > > ", "");
                                    line = line.Replace("template< class PathAlike >", "");
 
                                    if (line.Length > 0)
                                    {
                                        writer.WriteLine(line);
                                    }
 
                                }
                            }
                        }
                    }
                }
            }
            FilterFiles(pkgsCreated);
        }

And here is a quick and dirty way to find private and protected members:

private void FilterFiles(List<string> fileList)
        {
            foreach (string str in fileList)
            {
                var fileName = str.Replace(".temp", "");
                using (StreamWriter writer = new StreamWriter(fileName))
                {
                    using (StreamReader reader = new StreamReader(str))
                    {
                        while (!reader.EndOfStream)
                        {
                            string line = reader.ReadLine();
                            if (line.Trim() == "private:" ||
                                line.Trim() == "protected:")
                            {
                                writer.WriteLine(FilterPrivate(reader));
                            }
                            else
                            {
                                writer.WriteLine(line);
                            }
                        }
                    }
                }
 
                // Delete temp file
                FileInfo fi = new FileInfo(str);
                if (fi.Exists)
                    fi.Delete();
            }
        }
 
        private string FilterPrivate(StreamReader reader)
        {
            // Read through the stream until the private or protected members are skipped.
            while (!reader.EndOfStream)
            {
                string line = reader.ReadLine();
                if (line.Trim() == "public:")
                {
                    return "public:";
                }
                else if(  line.Trim() == "};")
                {
                    return "};";
                }
            }
            return string.Empty;
        }

I still have problems with sub classes, but I don’t need 100% API functionality for Lua, so for now I am just excluding them. I’ve got this process down so that I can take 200+ header files, run it through the tool, and execute the command line examples, and have outputted C++ code ready to include in my project without any problems.

Using ToLua++ can be a challenge at first, but once you have a good pipeline down, you’ll be able to make changes to your API, press a few buttons, and those changes can be reflected quickly and correctly in your Lua wrappers.

Here is just a quick example output of ToLua: (it’s not pretty)

/* method: SetYAxisFromTerrain of class  Core::CPlayer */
#ifndef TOLUA_DISABLE_tolua_Core_Core_CPlayer_SetYAxisFromTerrain01
static int tolua_Core_Core_CPlayer_SetYAxisFromTerrain01(lua_State* tolua_S)
{
 tolua_Error tolua_err;
 if (
     !tolua_isusertype(tolua_S,1,"Core::CPlayer",0,&tolua_err) ||
     !tolua_isnumber(tolua_S,2,0,&tolua_err) ||
     !tolua_isnoobj(tolua_S,3,&tolua_err)
 )
  goto tolua_lerror;
 else
 {
  Core::CPlayer* self = (Core::CPlayer*)  tolua_tousertype(tolua_S,1,0);
  float elapsedTime = ((float)  tolua_tonumber(tolua_S,2,0));
#ifndef TOLUA_RELEASE
  if (!self) tolua_error(tolua_S,"invalid 'self' in function 'SetYAxisFromTerrain'", NULL);
#endif
  {
   self->SetYAxisFromTerrain(elapsedTime);
  }
 }
 return 0;
tolua_lerror:
 return tolua_Core_Core_CPlayer_SetYAxisFromTerrain00(tolua_S);
}
#endif //#ifndef TOLUA_DISABLE

Ogre Paged Terrain and ODE

Posted on by
Reply

I have been working on trying to get Ogre’s new paged terrain to work with ODE for awhile now. At one point I just gave up and moved on to PhysX, but after working with PhysX, I realized getting control of specific collision events is difficult to impossible. I designed the Core engine using the strategy design pattern in many places, and using physics is one of them, so it is easy for me to switch from physics engines, GUI libraries, audio libraries, etc. After being fed up with PhysX, I moved back to ODE just for the fun of it. I like the comparative simplicity compared to Bullet and PhysX. Well this weekend I decided that I am going to fix this paged terrain issue once and for all. Finally, after forcing myself to work on it almost non-stop, I got it working, so here are my results.

I first started using ODE’s callback method like so:

PagedTerrainGeometry::PagedTerrainGeometry(World *world, Space* space, TerrainGroup* terrainGroup) 
	: Geometry(world, space),
	maxHeight(0),
	_terrainGroup(terrainGroup)
{
	TerrainGroup::TerrainIterator ti = terrainGroup->getTerrainIterator();
	while(ti.hasMoreElements())
	{
		Terrain* t = ti.getNext()->instance;
		if(t)
		{
			maxHeight = t->getMaxHeight();
			auto data = t->getHeightData();
			auto size = t->getSize();
			const f32 minHeight = t->getMinHeight();
			const f32 worldSize = t->getWorldSize();
			auto convertedData = new f32[size * size];		
 
			dHeightfieldDataID heightid = dGeomHeightfieldDataCreate();
			dGeomHeightfieldDataBuildSingle (heightid,
                                      convertedData,
                                      0,				// bCopyHeightData 
                                      worldSize, worldSize,
                                      size, size,
                                      scale,			// Scale
									  0,				//Offset
									  10,				// Thickness
									  0);				// Wrap
 
			dGeomHeightfieldDataBuildCallback(	heightid, //getSpaceID(space), 
												this, // pUserData ?
												PagedTerrainGeometry::_heightCallback, 
												(dReal) (worldSize), //X
												(dReal) (worldSize), //Z
												size, // w // Vertex count along edge >= 2
												size, // h // Vertex count along edge >= 2
												1.0,     //scale
												0.0,	// vOffset
												10.0f,	// vThickness
												0); // nWrapMode
 
			// Give some very bounds which, while conservative, makes AABB computation more accurate than +/-INF.
			dGeomHeightfieldDataSetBounds( heightid, minHeight,  maxHeight );
			_geom = dCreateHeightfield( getSpaceID(space), heightid, 1);
 
			_listener = 0;
			registerGeometry();
			setOrientation(Quaternion::ZERO);
		}
	}
}

The meat of it is here:

dGeomHeightfieldDataBuildCallback(	heightid, //getSpaceID(space), 
												this, // pUserData ?
												PagedTerrainGeometry::_heightCallback, 
												(dReal) (worldSize), //X
												(dReal) (worldSize), //Z
												size, // w // Vertex count along edge >= 2
												size, // h // Vertex count along edge >= 2
												1.0,     //scale
												0.0,	// vOffset
												10.0f,	// vThickness
												0); // nWrapMode

I defined my callback “_heightCallback”, and created my callback method:

dReal PagedTerrainGeometry::_heightCallback(void* data,int x,int z)
{
	PagedTerrainGeometry * const terrain = (PagedTerrainGeometry*)data;    
	if (terrain->_listener)
	{
		return static_cast <dReal> (terrain->_listener->heightAt(Vector3((f32)x, terrain->maxHeight, (f32)z)));
	}
    else
    {
        return static_cast <dReal> (terrain->getHeightAt(Vector3((f32)x, terrain->maxHeight, (f32)z)));
    }
}

When I ran it I noticed that my objects all hit at Y axis 85.XXX. After breaking in the callback I saw that ODE only requested the Y axis at the same points in my scene, no matter where the object was. The object could have been at vector(100, Y, 100), and it would still ask for the height at vector(76, Y 75), it was weird.

I didn’t feel like stepping through the ODE code, so I decided to using ODE’s other method of setting up a hightmap, and that is with the “dGeomHeightfieldDataBuildXXX” functions. In my case Ogre has the terrain data in an array of floats so the method I would use is “dGeomHeightfieldDataBuildSingle”.

The method call would look something like this:

dHeightfieldDataID heightid = dGeomHeightfieldDataCreate();
dGeomHeightfieldDataBuildSingle (heightid,
                            convertedData,
                            0,				// bCopyHeightData 
                            worldSize, worldSize,
                            size, size,
                            scale,			// Scale
							0,				//Offset
							10,				// Thickness
							0);				// Wrap

This is saying, don’t use my own callback, here is the data, you handle finding the Y axis for me. But when I run it, objects where rolling all over the place, and I couldn’t find a patterned as to why. The only thing I could think of was ODE expected the data in one format, and Ogre organized the data in another format. Turns out I was right, Ogre organizes the data for rendering efficiency, and needs to be ordered differently for ODE to except (Bullet too).

Here is my updated code for getting and organizing the data needed for ODE:

maxHeight = t->getMaxHeight();
auto data = t->getHeightData();
auto size = t->getSize();
const f32 minHeight = t->getMinHeight();
const f32 worldSize = t->getWorldSize();
auto convertedData = new f32[size * size];		
 
for(auto i = 0; i < size; i++)
{
	memcpy(convertedData + size * i, data + size * (size - i - 1), sizeof(f32) * size);
}
 
auto scale = size / (size - 1);

The important piece is the for loop, this orders the data for what a typical physics engine would want, this is also the order PhysX and Bullet uses. I tested the change and it worked, finially!

Here is the complete method:

PagedTerrainGeometry::PagedTerrainGeometry(World *world, Space* space, TerrainGroup* terrainGroup) 
	: Geometry(world, space),
	maxHeight(0),
	_terrainGroup(terrainGroup)
{
	TerrainGroup::TerrainIterator ti = terrainGroup->getTerrainIterator();
	while(ti.hasMoreElements())
	{
		Terrain* t = ti.getNext()->instance;
		if(t)
		{
			maxHeight = t->getMaxHeight();
			auto data = t->getHeightData();
			auto size = t->getSize();
			const f32 minHeight = t->getMinHeight();
			const f32 worldSize = t->getWorldSize();
			auto convertedData = new f32[size * size];	
 
			for(auto i = 0; i < size; i++)
			{
				memcpy(convertedData + size * i, data + size * (size - i - 1), sizeof(f32) * size);
			}
 
			auto scale = size / (size - 1);
			dHeightfieldDataID heightid = dGeomHeightfieldDataCreate();
			dGeomHeightfieldDataBuildSingle (heightid,
                                      convertedData,
                                      0,				// bCopyHeightData 
                                      worldSize, worldSize,
                                      size, size,
                                      scale,			// Scale
									  0,				//Offset
									  10,				// Thickness
									  0);				// Wrap
 
			// Give some very bounds which, while conservative, makes AABB computation more accurate than +/-INF.
			dGeomHeightfieldDataSetBounds( heightid, minHeight,  maxHeight );
			_geom = dCreateHeightfield( getSpaceID(space), heightid, 1);
 
			_listener = 0;
			registerGeometry();
			setOrientation(Quaternion::ZERO);
		}
	}
}

For more information on this check out this Ogre forum post here

Linq and Sql Server Sprocs

Posted on by
Reply

For me I like to use Linq on projects that don’t have a defined data access layer, so small to medium projects I tend to lean towards Linq to Sql. I’ve found that there are still developers out there that are not aware of the fact that you can use Linq to Sql with Sql Server stored procedures. This is a powerful feature you can use so I thought I would give a quick example of accessing a sproc using Linq to Sql.

Let’s say you have a simple procedure that gets the user name and the access level of that user:

CREATE PROCEDURE [dbo].[GetUser]
@UserName AS nvarchar(50),
@Password AS nvarchar(MAX)
AS
BEGIN
SET NOCOUNT ON;
 
SELECT UserName, AccessLevel
FROM Users
WHERE Users.UserName = @UserName AND
Users.Password = @Password AND
Users.ActiveAccount = 1 AND
(Users.AccountExpires &gt;= SYSDATETIME() OR
Users.AccountExpires IS NULL)
 
END

Pretty simple, OK, let’s move on to setting linq up. You need to add a new dbml file, and once you open it up in the pretty GUI editor you should see a blank white file. What I do is connect to my Sql DB in VS and from there you are able to drag and drop the tables, and stored procedures that you want into that editor. Simple rigth? Here is what you should see once you add your sproc:

Now let’s move on to our code:

Dim dc As New MyDataContext
Dim LocalUser = dc.GetUser(UserName, Password).Single()

Wow, nice and easy, now have fun.

Lambdas, C++, and why use it?

Posted on by
Reply

When using .NET and specifically using Linq, I tend to utilize lambdas often, or at least when the need for a quick and simple anonymous function needs to be used.  But the new C++0x standard adds lambda expression support to, and being a user of Visual Studio 2010, I know its already supported.  In fact I converted an existing project I’ve been working on for a few years now from 2008 to 2010 so that I could take advantage of just a few of the new features added in 2010 that supports the C++0x standard.  I didn’t convert my huge solution to 2010 for the use lambdas though, why would anyone need to use it?

After doing some investigation into the uses, I’ve narrowed down the common uses of lambdas in C++ to mainly helping you with stl functions like “for_each”, and “find_if”.  The great thing (or maybe a bad thing) about C++ is it allows you a plethora of ways to blow your foot off, and being the stickler for optimization that I am, I don’t usually use the for_each method and many other methods in the stl library that make your life easier, so the need for lambdas in my native life are all but non-exist.  That is until saw Microsofts PPL examples here.  The examples of how we can use lambda expressions to pass to different parallel processes looks kind of cool, but why use PPL when you can use OpenCL?

Fun with HTML 5 Datetime

Posted on by
Reply

I haven’t had much time messing with HTML 5 yet, only reading up on it, and it looks great. The browser support for it seems to be slowly coming to standard with Chrome and Firefox leading the way, and IE as the lone straggler. One of the many new features that I like is the built in “datetime” type for the input control, so here are my results with that control using Chrome.

Doing a simple:

<!DOCTYPE HTML>
<html>
<body>
<input type="datetime" />
</body>
</html>

I got just a blank control with some arrows. You’re able to modify the date by the arrows of course, but you can also use the scroll mouse too, so that’s cool.

Adding the min and max attributes didn’t seem to do what I expected because I can type “2000″ for the year and it would accept it.

<input type="datetime" min="1/1/2010" max="1/1/2012" />

After Messing around with these attributes I found that you need to have the format like this: YYYY-MM-DD

<input type="date" min="2010-01-01" max="2011-01-01" />

That seems to work nicely.

Integrating OpenSteer with Ogre

Posted on by
Reply

In my journey to the perfect 3D simulation API I ran into the problem of AI, and the lack of time, and/or skill to implement it. Along came OpenSteer, actually OpenSteer has been out for quite a while and is a great library for agent steering and basic decision making. The term “steering” is just that, pointing the agent in the correct direction and moving it. That may seem simple, but remember you have obstacles to maneuver around (3D at that), other agents that may be chasing you, pathway points you need to follow, etc. So that was my reason to go with OpenSteer, its got a lot of what you need to get a simulation going, both in 2D and 3D, and it is some what tested.

One thing that people tend to have a problem with is how does it work with my engine, or in my case Ogre. To answer that, you first need to know how OpenSteer works, but remember with any library, you are going to give it an input, and it’s going to give you an output. Same things with any other library, you provide it an input, it does its magic, and you get back an output.

To better wrap your head around this, you need to understand what you want from OpenSteer. You don’t want it to control your model, you want it to give you a direction and speed, and that’s about it.

Here’s an example of the “Wander” state, basically an agent is wandering around the world.

// Get obs list
Vector<AI::Obstacle*> obs;
CGameObjectManager::Instance()->GetSphereObstacles(obs);
 
// Get walls
Vector<AI::Wall*> walls;
CGameObjectManager::Instance()->GetWalls(walls);
 
reVector3Df avoidance = reVector3Df::ZERO;
reVector3Df steer = reVector3Df::ZERO;
 
avoidance = m_Player->GetAiVehicle()->steerForWallAvoidance(walls);
avoidance += m_Player->GetAiVehicle()->steerToAvoidOabstacles_QuickSphere(obs);
if (avoidance == reVector3Df::ZERO)
{
	int intTurningness = 1;
	reVector3Df wander2d = m_Player->GetAiVehicle()->steerForWander(elapsedTime);
	wander2d.y = 0;
	steer = m_Player->GetAiVehicle()->forward() + (wander2d * intTurningness);	
	CORE_TEXT("non_avoidance", "nonavoidance:" + StringConverter::toString(steer));
}
else
{
	steer = avoidance;
	steer.y = 0;
	CORE_TEXT("avoidance", "avoidance:" + StringConverter::toString(steer));
}
m_Player->GetAiVehicle()->ApplySteeringForce (steer, elapsedTime);

Now, the first thing I do is get a list of my walls, and obstacles in the world. Every “Player” in Core has its own “Vehicle”. This is what an agent is in OpenSteer, so whenever I need to steer the player, I use its own instances of OpenSteers version of an agent. So I call “steerForWallAvoidance” from the player object.

These two lines basically asks OpenSteer if I am going to run into anything:

avoidance = m_Player->GetAiVehicle()->steerForWallAvoidance(walls);
avoidance += m_Player->GetAiVehicle()->steerToAvoidOabstacles_QuickSphere(obs);

If not then I am going to basically get a random direction to go:

reVector3Df wander2d = m_Player->GetAiVehicle()->steerForWander(elapsedTime);
steer = m_Player->GetAiVehicle()->forward() + (wander2d * intTurningness);

Other wise I am going to steer to avoid the object in my way. Either way I have a direction in the form of a 3D vector, and I move the player along that vector:

m_Player->GetAiVehicle()->ApplySteeringForce (steer, elapsedTime);

Boom, done! And this is ran almost every frame is not every frame.

Note, “steerForWallAvoidance” and “steerToAvoidOabstacles_QuickSphere” are additions I added to OpenSteer, in fact for me, its a completely different library because I’ve modified it so much. Also, it won’t return Ogre 3D vectors, it has its own vector it will return, but I also have my own Vector library “reVector3Df”, so I’ve modified both Ogre’s and OpenSteer’s source to only use mine.

.NET is Great for Content Tools

Posted on by
Reply

I don’t think of Core as a API for game, but of an API for 3D simulations. There’s a difference, but I won’t get into that. Just like any web application, client side CRM tool, Geo mapping program, and any other modern and somewhat complex application in developed now, 3D simulations need to be data driven as well. The data that a 3D simulation or a game uses usually differs quite a bit as far as what the data is, and how it is processed. Most games need to run at optimal performance (which is why most are still developed in C/C++), so the data location, type, and organization of the data can be very different from a typical ASP.NET app.

In the simple demos that I can making now, I already have over a gig of content in some of these demos, so organizing them and ensuring the data is there can be an complex task sometimes. So as my problem becomes more complex, so does my solution, and that solution is a series of tools. The primary tool I use to organize my content is a custom C# app that I started about 6 months back and it has already saved me a lot of time.

C# and .NET is general is a great technology to use, and that includes working with native C++ APIs. Here’s some of the features I’ve added to my program I called “Asset Manager” (makes sense).

1. Copying assets. In my SVN repository, I have a lot of models, textures, materials, animations, etc in a number of different directories. And when you start buying pre-packaged models and export them, you tend to loose track of where they all are. By copying all the assets from a root directory, I can output all the content into one single directory, pretty simple.

FileTypes.Add("tga");
FileTypes.Add("png");
FileTypes.Add("bmp");
FileTypes.Add("jpg");
FileTypes.Add("tif");
FileTypes.Add("psd");
FileTypes.Add("dds");
FileTypes.Add("mesh");
FileTypes.Add("cg");
FileTypes.Add("program");
FileTypes.Add("source");
FileTypes.Add("skeleton");
FileTypes.Add("material");
FileTypes.Add("particle");
FileTypes.Add("ogg");
 
foreach (string d in Directory.GetDirectories(sDir))
{
    if (!d.Contains(".svn"))
    {
        foreach (string f in Directory.GetFiles(d))
        {
            FileInfo fi = new FileInfo(f);
            if (FileTypes.Contains(fi.Extension.Replace(".", "")))
                FileList.Add(f);
        }
    }
    DirSearch(d);
}
 
// Copy files to out directory
foreach (string f in FileList)
{
    FileInfo fi = new FileInfo(f);
    if (!File.Exists(txtOutDir.Text + "\\" + fi.Name))
    {
        File.Copy(f, txtOutDir.Text + "\\" + fi.Name);
    }
    else
    {
        AddReport("*** File: " + f + " already exist.***");
        errors = true;
    }
}

There’s more to is, but you get the idea.

2. Material Validation: I have an option to merge all my material files into one big one. The original idea behind that is of course less files are opened and closed during Ogre’s initialization, but it turns out that Ogre has a material line limit, and I haven’t figured out a solution for this yet. But I also learned that I had multiple materials with the same name, and once Ogre finds a material with the same name, it just stops parsing every other material in the file! So using my existing functionality, I added a quick check to report back to me whenever if found multiple material names, problem solved.

private void btnMatRun_Click(object sender, EventArgs e)
{
    txtMatReport.Text = string.Empty;
    MatList.Clear();
 
    if (txtMatRoot.Text.Length == 0)
    {
        MessageBox.Show("Enter an output directory retard.");
        return;
    }
 
    List<string> list = new List<string>();
    DirSearch(txtMatRoot.Text, "*.material", ref list);
    foreach (string s in list)
    {
        string ret = ValidateMaterial(File.ReadAllLines(s));
        if (ret.Length > 0)
            txtMatReport.Text += ret + ": " + s + "\r\n";
    }
 
    MessageBox.Show("Done.");
}
 
string ValidateMaterial(string[] Content)
{
    string ret = string.Empty;
    List<string> matNames = new List<string>();
    foreach (string s in Content)
    {
        if (s.Trim().ToLower() == "material")
        {
            // No material name, invalid
            ret += "No material name\r\n";
        }
        else if (s.Contains("material"))
        {
            if (matNames.Contains(s))
                ret += "Material name already declared: " + s + "\r\n";
            else if (matNames.Contains("}material"))
                ret += "Materal started with no return." + s + "\r\n";
            else
                matNames.Add(s);
        }
    }
    return ret;
}

3. Xml Generation: Just like 99% of modern applications out there use Xml for custom settings, so does Core. A section in the application config specifies which assets should be loaded initially. For now I am loading everything because it is difficult to initialize your resource group manager, then add new materials after that, it won’t parse it. .NET has some great support for generating Xml, so bonus!

4. Profiles Xml Generation: I have the same functionality to create Xml for inventory items. I’ll explain the design of my inventory items later on, but the Asset Manager is able to create the Xml you need to load it in the API.

5. C++ Code Generator: Last but not lease I just added a new feature that will go through the output directory when copying all your assets (in #1), and generating a C++ class for every “game object” that is being rendered in the simulation. I will talk about this design later, but the API doesn’t just render a 3D model, it needs to be encompassed with meaning, and that defined meaning inherits from a common abstract class “CGameObject”. As of today I have 209 3D models, and in less then a second I can generate 8200 lines of code that will allow me to import those models in the API.

Its rough, but here’s an example:

// If the output directory doesn't exsit, create it
DirectoryInfo outputInfo = new DirectoryInfo(outputDir);
if (!outputInfo.Exists)
{
    outputInfo.Create();
}
 
DirectoryInfo dirInfo = new DirectoryInfo(assetDirectory);
List<FileInfo> fileInfoList = new List<FileInfo>();
foreach(FileInfo fileInfo in dirInfo.GetFiles())
{
    if (fileInfo.Extension == ".mesh")
    {
        CreateGOPHeader(fileInfo, outputDir);
        CreateGOPSource(fileInfo, outputDir);
        fileInfoList.Add(fileInfo);
    }
}
CreateGOPFactoryExportHeader(fileInfoList, outputDir);
CreateGOPFactoryExportSource(fileInfoList, outputDir);
 
.
.
.
 
private void CreateGOPHeader(FileInfo fileInfo, string outputDir)
{
    string className = GetClassName(fileInfo);
    StreamWriter streamWriter = new StreamWriter(outputDir + "\\" + className + ".h");
    streamWriter.WriteLine("#ifndef __" + className.ToUpper() + "_H__");
    streamWriter.WriteLine("#define __" + className.ToUpper() + "_H__");
    streamWriter.WriteLine();
    streamWriter.WriteLine("#include \"Defines.h\"");
    streamWriter.WriteLine("#include \"IGameObject.h\"");
    streamWriter.WriteLine("#include \"Plugins/IGameObjectFactory.h\"");
    streamWriter.WriteLine();
    streamWriter.WriteLine("namespace Core");
    streamWriter.WriteLine("{");
    streamWriter.WriteLine("namespace Plugin");
    streamWriter.WriteLine("{");
    streamWriter.WriteLine("\tclass " + className + " : public IGameObject");
	streamWriter.WriteLine("\t{");
    streamWriter.WriteLine("\tpublic:");
    streamWriter.WriteLine("\t\t" + className + "(const char* Name, reVector3Df& Pos);");
    streamWriter.WriteLine("\t\t~" + className + "();");
    streamWriter.WriteLine();
    streamWriter.WriteLine("\t\tvoid Update(const f32& elapsedTime) { }");
    streamWriter.WriteLine();
    streamWriter.WriteLine("\t};");
    streamWriter.WriteLine();
    streamWriter.WriteLine("\tclass " + className + "Factory : public IGameObjectFactory");
	streamWriter.WriteLine("\t{");
    streamWriter.WriteLine("\tpublic:");
    streamWriter.WriteLine("\t\t" + className + "Factory();");
	streamWriter.WriteLine("\t\t~" + className + "Factory();");
    streamWriter.WriteLine();
    streamWriter.WriteLine("\t\tIGameObject* CreateObject(const char* Name, reVector3Df& Pos);");
    streamWriter.WriteLine("\t};");
    streamWriter.WriteLine("}");
    streamWriter.WriteLine("}");
    streamWriter.WriteLine();
    streamWriter.WriteLine("#endif // __" + className.ToUpper() + "_H__");
    streamWriter.Close();
}
// Create the optional project
CreateGOPProject(projectName, fileInfoList, outputDir);

.NET has functionality that will allow you to generate code, but I don’t think it does for C++, plus its easier just to write it out manually to a file.

There you go, nothing too advanced but maybe it will give you some ideas on how you can streamline your content creation process.

A little Overview on REST

Posted on by
Reply

REST (Representational State Transfer) is becoming more and more of a buzz word, but I’ve noticed that not a lot of developers know what it is, or even have a basic understanding of it. First and for most, REST is not new, but it seems that it’s popularity is. Kind of like how AJAX was out there for a long time but didn’t become popular until Google Maps started using it, then everyone started using it. But REST is really a new word to describe an old technology. In general REST is more of an coding style, or architecture more then a standard or protocol.

As a big fan of WCF and have plenty of experience with it, I’ll describe REST and using it with WCF. As you may or may not know, with WCF you can use different types of endpoints, and even create your own endpoints. When WCF came out in 2006, it didn’t by default support REST, but you could extend WCF to support it. I’ve never done it, and didn’t need to at the time so I won’t go into that.

A few things about SOAP. SOAP is a wonderful standard for communications over the web and WCF uses it rather great. The thing with SOAP is it is a big and complex standard that can be a pain to follow sometimes. Now one of the cool things with WCF, even version 1 was it had a lot of tools included that dealt with SOAP and WSDL complexities, but it can still be problematic sometimes.

The thing with REST is it is much simpler then SOAP, and you don’t need to worry about many of the complexities that comes along with SOAP. Messages that come along the wire are much smaller, a lot less serialization, there’s less work on the client side, you can address everything using URIs, and there is no contract, no WSDL, isn’t life good?

Setting up REST in WCF:
1. Under your behaviors you need to add the REST behavior called “webHttp”
2. In the binding attribute in your service tag you need “webHttpBinding”
Note: There are a few templates you are able to use in Visual Studio that will set this up for you.

This is of course all all you need to do, but your endpoints are setup. Microsoft has plenty of documentation on this subject that I don’t need to reproduce. here

Zipping in .NET

Posted on by
Reply

If you want to compress something in .NET, you are provided with a simple little namespace with some simple little functionality to compress your target files in compressed output. Using this code you can create a gzip output:

DirectoryInfo directoryInfo = new DirectoryInfo(directory);
foreach (FileInfo fileInfo in directoryInfo.GetFiles())
{
   using (FileStream inFile = fileInfo.OpenRead())
   {
      using (FileStream outFile = File.Create(fileInfo.FullName + ".gz"))
      {
         using (GZipStream Compress = new GZipStream(outFile,             CompressionMode.Compress))
         {
            inFile.CopyTo(Compress);
         }
      }
   }

That’s all fine and dandy but it’s gzip, and not zip. If you want to open your output, you need to download another tool to get the files back (or code it ourself). You would think .NET would come with zip format functionality too, but it doesn’t. If you google .NET zip, you will find a number of 3rd libraries that will help you out, and even some with the nerve to even charge you for there library.

Using Ionic from codeplex here, I was able to quickly compress all the contents of a directory into a nice neat little zip file using even simpler code:

using (ZipFile zip = new ZipFile())
{
     DirectoryInfo directoryInfo = new DirectoryInfo(directory);
     foreach (FileInfo fileInfo in directoryInfo.GetFiles())
     {
          zip.AddFile(fileInfo.FullName, ".\\");
     }
     zip.Save(directory + "\\zzOut.zip");
}

And hazah, even less code, and we have a zip file that an end user can open in Windows without having to download 7zip.