Creating a multiplayer browser-based game engine

Discussion in 'Off Topic' started by Jephir, Jul 28, 2016.

  1. Jephir

    Jephir ALL GLORY TO THE JEPHIR

    Messages:
    1,409
    Likes Received:
    6
    Trophy Points:
    0
    Hey guys. I haven't been active on the Empires forum for a while but I'd like to share something that I've been working on.

    I like to do personal projects in the summer and my main interest in the past few years has been creating my own game engine. This mostly came out of my frustration after developing on Source and UDK. Often, I've complained about impossible to debug crashes, missing core documentation, and general over-complexity of current game engines. So I decided I would do something about it, and create a game engine that I would actually enjoy using!

    My initial vision was a 3D multiplayer browser-based engine. I like the browser platform because people can play your game without having to download or install a client.

    Last summer, I read many books on engine architecture and 3D graphics and created my own 3D renderer from scratch. It took a lot of time but eventually I figured out how to draw 3D models just from the raw vertex data and texture maps.

    Everything went well until I tried getting animations working.

    [​IMG]

    Unlike static models, animations are a huge pain. The main problem is that it's very time-consuming to debug animation errors. When something goes wrong, the whole model gets clobbered and you have to spend huge amounts of time checking your matrices, exporters, and rendering code to see where the problem is.

    In the end, I spent an entire summer doing 3D rendering and I couldn't get animations working. I took a hard look at what I was doing and decided that 3D was not worth it, at least in the short run. My goal is to create a game engine, not a renderer after all, and I was spending too much time on this one problem.

    This summer, I'm recoding the engine to use 2D graphics. The difference between 3D and 2D development is staggering. I literally got animations working in one day. I should have done 2D from the start, but at least I learned from my mistake.

    [​IMG]

    It's going to take a while to translate all the 3D code to 2D, however, it feels good to finally be making progress again. If you worked with me on Factions you know that I don't like to share progress until I have an actual product working. However, I think I need to change things up, because when I don't share my work, I often get stuck on inconsequential problems instead of making real progress. So I'm doing things differently this time and releasing updates as I go.

    The engine is called Sagaworld and you can access it at

    https://saga.world


    Note that the current release is very basic and you can only move your character and place tiles. It will take some time to port all the old code to the new engine.

    I'll be posting updates on Twitter at

    https://twitter.com/sagadotworld

    I have just over a month left before fall classes start so I need to get motivated to get things done. Thanks for reading this massive wall of text. I've been working on this for a while and hopefully it will turn into a useful, productive engine for anyone to use!
     
    Devourawr and Varbles like this.
  2. Paradox

    Paradox I am a gigantic asshole who loses people's hard wo

    Messages:
    6,926
    Likes Received:
    148
    Trophy Points:
    0
    Wow, I've always respected you as a player and a person Jephir but this tops it.
    This is some real stuff you're working on, congratulations and I wish you the best with the rest.
     
  3. f1r3w4rr10r

    f1r3w4rr10r Modeler

    Messages:
    2,475
    Likes Received:
    4
    Trophy Points:
    0
    Really nice going there! Keep it up!
     
  4. complete_

    complete_ lamer

    Messages:
    6,437
    Likes Received:
    144
    Trophy Points:
    0
    [​IMG]
    neat!
     
    Jephir likes this.
  5. Jephir

    Jephir ALL GLORY TO THE JEPHIR

    Messages:
    1,409
    Likes Received:
    6
    Trophy Points:
    0
    Nice. I'm currently working on getting the overworld server up so that changes are persistent. Right now it runs a local server on your client, so changes are not persisted yet.
     
  6. blizzerd

    blizzerd Member

    Messages:
    10,552
    Likes Received:
    60
    Trophy Points:
    0
    cool shit
     
  7. Jephir

    Jephir ALL GLORY TO THE JEPHIR

    Messages:
    1,409
    Likes Received:
    6
    Trophy Points:
    0
    It look longer than expected but I finally finished the basic netcode for the new engine.

    [​IMG]

    The basic idea that separates this engine from all the others that I've used in the past is that the entire game state is held in one object called WorldState. This is a plain JSON object that holds the information of every game object in the world. In contrast, most other engines use C++ classes to hold game object information, like Source's CEntity.

    The problem with using classes in networked games is that you have to serialize all the entity information into an update packet, sync this update packet to the server and clients, and then restore the entity information on the receiver. In practice, this is very difficult to do because you have to sync the internal state of all your engine components (graphics, physics, sound), whenever you get new entity information. Mistakes here lead to problems that are difficult to replicate and seemingly "random" networking bugs that only occur under specific circumstances.

    This engine takes the "reverse" approach. We start with the data structure first. All the engine components read WorldState every frame and update their internal state. For example, the Graphics component checks if an object in WorldState has a Sprite property, and then spawns a sprite in the internal graphics library. The component then updates the position of the library sprite to match the WorldState information every frame. If the Sprite property is deleted, then the graphics components deletes the corresponding library sprite.

    The key is that every engine subsystem can regenerate it's internal state solely from WorldState. This means that networking is now trivial. All that needs to be done is to broadcast the WorldState object to each client, and they can regenerate their internal state to match the world state. If a problem occurs, it is easy to debug, because all you have to look at is WorldState - all the engine components match their internal state with this single data structure.

    The engine website https://saga.world has been updated with the new netcode. I haven't finished migrating everything yet, however, so player animations and tile creation are temporarily disabled. I'm working on re-implementing those under the new system.
     
    Last edited: Aug 8, 2016
  8. blizzerd

    blizzerd Member

    Messages:
    10,552
    Likes Received:
    60
    Trophy Points:
    0
    bonkers, can we play infiltration on this?
     
  9. Deiform

    Deiform Member

    Messages:
    2,491
    Likes Received:
    10
    Trophy Points:
    0
    How are you handling data instancing so that clients only receive what they need and not the entire WorldState?
     
  10. Jephir

    Jephir ALL GLORY TO THE JEPHIR

    Messages:
    1,409
    Likes Received:
    6
    Trophy Points:
    0
    There will be a server class called Replicator that listens for updates to WorldState and handles sending the appropriate state to clients. It has two main rules for determining what to send.

    The first is a spatial grid that is part of the internal state of Replicator. Every game object in WorldState is assigned to a grid. A client is then only eligible to receive world state in the current grid and adjacent grids of it's current view (usually the player's currently controlled game object). This filters out game objects that are too far away from the player's current view.

    The second are condition-based rules that the world creator specifies in the game editor. You can create conditions like "If a client is on the same team as a unit, then send them the Health and Mana property of that unit". This allows you to control what parts of the WorldState that a client receives.
     
  11. Jephir

    Jephir ALL GLORY TO THE JEPHIR

    Messages:
    1,409
    Likes Received:
    6
    Trophy Points:
    0
    Persistence is in! Now all changes to the world will be saved. There is basic "prop protection" so you can't overwrite other people's tiles. Basically you can only build on the black space or edit your own tiles.

    https://saga.world
     
  12. Fricken Hamster

    Fricken Hamster Mr. Super Serious

    Messages:
    3,620
    Likes Received:
    2
    Trophy Points:
    0
    Browser networking sucks.
    The only hope is webRTC, but thats still no where near ready, or doing a lot of clientside prediction, which only really works with mmorpgs.

    Artillery games tried to do live multiplayer game on the browser, but they had no plan and realized it sucked. So now they just have shitty generic game thats going to go nowhere.

    Using json is pretty wasteful. I send binary over websockets. Its pretty nice.
     
  13. Jephir

    Jephir ALL GLORY TO THE JEPHIR

    Messages:
    1,409
    Likes Received:
    6
    Trophy Points:
    0
    There's some massive lag right now on the server. Trying to figure out what's causing it.
     
  14. Trainzack

    Trainzack Member

    Messages:
    384
    Likes Received:
    4
    Trophy Points:
    0
    [​IMG]
    Messed around with it a bit. Worked okay, except movement always uses the direction from the previous click, and would randomly slide around for no reason, and I couldn't figure out how to delete things. But other than that. it seemed to work okay.

    As an aside, if you ever find yourself wanting to make animations work again, this guy ran into what looks to be the same problem, and his solution might help you?
     
  15. Jephir

    Jephir ALL GLORY TO THE JEPHIR

    Messages:
    1,409
    Likes Received:
    6
    Trophy Points:
    0
    Awesome, thanks for helping to test it out. The movement issue was probably caused by the client clock getting desynchronized from the server clock. I pushed an update that adds clock synchronization which should fix the issue now.

    The delete tool (and code tool) currently doesn't do anything, although I plan on implementing them soon.
     
  16. Trainzack

    Trainzack Member

    Messages:
    384
    Likes Received:
    4
    Trophy Points:
    0
    Movement seems to work as intended. However, when I went back to check it out, I noticed that everything wasn't quite how I left it:
    [​IMG] (The _A below is just something I stated to add before I noticed).

    Either not all of the tiles are there, or they are in the wrong draw order. I suspect the latter.
     
  17. Jephir

    Jephir ALL GLORY TO THE JEPHIR

    Messages:
    1,409
    Likes Received:
    6
    Trophy Points:
    0
    Yeah, the sprite depth is lost when restoring the WorldState from the database. However, the creation time of each sprite is recorded, so it's fixable. I just have to implement sorting.
     
  18. Trainzack

    Trainzack Member

    Messages:
    384
    Likes Received:
    4
    Trophy Points:
    0
    Just checked, looked sorting works fine now!

    Also, I danced around someone else. I think they got unnerved and left.
     
  19. Trainzack

    Trainzack Member

    Messages:
    384
    Likes Received:
    4
    Trophy Points:
    0
    [​IMG]
    Yeah, so it turns out that you can select things outside of the palette.
     
  20. Jephir

    Jephir ALL GLORY TO THE JEPHIR

    Messages:
    1,409
    Likes Received:
    6
    Trophy Points:
    0
    Oh god, how did that happen.

    Anyway, I'm currently working on optimizing the network code. There's a long delay when joining the server at the moment. The next few updates may just be patches (no user-visible features).

    Edit: Actually it might not be the networking code. The initial update only takes about 100 ms. I think it's loading the graphics tiles that are causing the lag. The tiles look screwed up again.
     
    Last edited: Aug 25, 2016

Share This Page