27 Sept 2008

What's New in the September Nebula3 SDK


I finally got around to pack a new N3 SDK together. I'll upload it on Monday when I'm back in the office, in the meantime here's a rough What's New list. A lot of under-the-hood-stuff has changed, and I had to remove a few of the fancy front-end-features for now (for instance, the N2 character rendering had to be removed when I implemented the multi-threaded renderer, and the shader lighting code is broken at the moment). I'll care about this front-end stuff in the next release.

General Stuff


  • changes to enable mixing Nebula2 and Nebula3 code, mainly macro names are affected (DeclareClass -> __DeclareClass, ImplementSingleton -> __ImplementSingleton etc...)
  • started to remove #ifndef/#define/#endif include guards since pretty much all relevant compilers (VStudio, GCC, Codewarrior) support #pragma once 
  • moved identical Win32 and Xbox360 source code into a common Win360 namespace to eliminate code redundancies
  • added a new Toolkit Layer which contains helper classes and tools for asset export
  • added and fixed some Doxygen pages

Build System

  • re-organized VStudio solution structure, keeps all dependent projects in the same solution, so it's no longer necessary to have several VStudios open at the same time
  • it's possible now to import VStudio projects through the .epk build scripts (useful for actual Nebula3 projects which do not live under the Nebula3 SDK directory)
  • new "projectinfo.xml" file which defines project- and platform-specific attributes for the asset batch-export tools
  • split the export.zip archive into one platform-neutral and several platform-specific archives (export.zip contains all platform-independent files, export_win32.zip, export_xbox360.zip, export_wii.zip contain the platform-specific stuff)
  • added general multiplatform-support to the asset-pipeline (e.g. "msbuild /p:Platform=xbox360" to build Xbox360-assets)
  • new command-line build tools (with source):
    • audiobatcher3.exe (wraps audio export)
    • texturebatcher3.exe (wraps texture export)
    • shaderbatcher3.exe (wraps shader compilation)
    • buildresdict.exe (generates resource dictionary files)
    • these tools mostly just call other build tools (like xactbld3.exe, nvdxt.exe, or build tools for game-console SDKs)
  • note that the public N3-SDK only contains Win32 support for obvious legal reasons 

Foundation Layer

  • fixed thread-safety bugs in Core::RefCounted and Util::Proxy refcounting code
  • added WeakPtr<> class for better handling of cyclic references
  • added type-cast methods to Ptr<>
  • simplified the System::ByteOrder class interface
  • added platform-specific task-oriented "virtual CPU core id's" (e.g. MainThreadCode, RenderThreadCore, etc...)
  • added a System::SystemInfo class
  • added Threading::ThreadId type and static Threading::Thread::GetMyThreadId() method
  • proper thread names are now visible in the VStudio debugger and other debugging tools
  • SetThreadIdealProcessor() is now used to assign threads to avaible CPU cores on the Win32 platform
  • new HTTP debug page for the Threading subsystem (currently only lists the active Nebula3 threads)
  • MiniDump support: crashes, n_assert() and n_error() now write MiniDump files on the Win32 platform
  • new Debug subsystem for code profiling:
    • offers DebugTimer and DebugCounter objects
    • HTTP debug page allows to inspect DebugTimers and DebugCounters at runtime
  • new Memory::MemoryPool class for allocation of same-size memory blocks (speeds up allocation and reduces heap fragmentation)
  • some new and renamed methods in Math::matrix44
  • Http subsystem now runs in its own thread
  • added SVG support to Http subsystem (Http::SvgPageWriter and Http::SvgLineChartWriter)
  • added IO::ExcelXMLReader stream reader class, allows to read XML-formatted MS Excel spreadsheet files
  • added Behaviour mode to Messaging::AsyncPort, defining how the handler thread should wait for new messages:
    • WaitForMessage: block until message arrives
    • WaitForMessageOrTimeOut: block until message arrives or time-out is reached
    • DoNotWait: do not wait for messages
  • added Remote subsystem, allows remote-controlling N3 applications through a TCP/IP connection
Render Layer
  • moved rendering into its own thread (InternalGraphics subsystem on the render-thread side, and Graphics front-end subsystem on the main-thread side)
  • added CoreAnimation and Animation subsystems (under construction)
  • added UI subsystem for simple user interfaces (under construction)
  • added CoreAudio and Audio subsystems (under construction):
    • CoreAudio is the back-end and runs in its own thread
    • Audio is the "client-side" front-end in the main-thread (or any other thread)
    • designed around XACT concepts
    • comes with XACT wrapper implementation
  • added CoreGraphics::TextRenderer and CoreGraphics::ShapeRenderer classes, both intended for rendering debug visualizations
  • added debug rendering subsystem (currently under the Debug namespace)
  • Frame subsystem: FramePostEffects may now contain FrameBatches
  • Input subsystem: disconnected XInput game-pad slots now only check every 0.5 seconds for connected game-pads
  • Resources subsystem: added ResourceAllocator/ResourceLump system to prepare for true resource streaming on console-platforms
Application Layer and Addons:
  • removed CoreFeature (this stuff had to go into the GameApplication class to prevent some chicken-egg problems)
  • added NetworkFeature (under construction)
  • added UIFeature (under construction)
  • new CoreNetwork and Multiplayer addon wrapper subsystems for RakNet

Please note the special RakNet licensing conditions. Basically, RakNet is not free if used for a commercial project (http://www.jenkinssoftware.com/). Licensing details for 3rd party libs can be found on the Nebula3 documentation main page.

Stuff I want to do soon

  • fix the shader lighting code
  • add more shaders to bring the shader-lib up-to-par with N2
  • finish the CoreAnimation and Animation subsystems
  • design and implement proper skinned character rendering subsystem
  • add missing functionality to Audio subsystems (for instance sound categories) 
  • make shaders SAS compatible so they work with tools like FXComposer
  • implement a proper resource-streaming system on the 360 (as proof on concept)
  • optimize messaging (use delegate-mechanism for dispatching, optimize message object creation, add double buffering behaviour to AsyncPort for less thread-synchronization overhead)