A Go Microservice, first look – part 2

This post is the 2nd and final post in this series, and follows on from A Go Microservice, first look – part 1

Adding a datastore – MongoDB integration

To interact with MongoDB, we need to add the ‘Go MongoDB driver‘ (the client module). This is an external dependency, and our first for this project so far. We will add the official driver from Mongo using the go get tool from the command line.

go get go.mongodb.org/mongo-driver

This will download the driver and update the Go module file for this project, which now looks like this;

module github.com/mikej6502/todo-list-svc

go 1.16

require go.mongodb.org/mongo-driver v1.5.3 // indirect

Installing MongoDB

There are many options open to us on this one, MongoDB offer a free developer sandbox in the cloud which you can sign up for and use, so no need to install anything. For this demo, I am opting to stay local and use docker. Docker makes it ridiculously easy to install pretty much anything. We will run docker locally in a container and for now ignore volumes, this means all data will be lost once the container is destroyed, no good for production, but fine for demo purposes. Official image available from Docker Hub

docker run --name my-mongodb -d -p 27017:27017 mongo

Note: There is no security set up for this as this is only a demo running locally

As our Go application will connect over HTTP using a connection string, it really doesn’t matter which instance of MongoDB use you (cloud, local install or docker).

The Implementation

Replacing the ‘In Memory’ database with the new MongoDB is as simple as a creating a new concrete implementation of the DataStore interface. The code that consumes the datastore is accessing it via the public interface methods (it’s API), so simply adding a new instance for MongoDB and switching one for the other should do it.

A new Implementation of the DataStore Interface.

In part 1 we defined an interface called Datastore, and using this an InMemoryDataStore was created. Below is the existing interface for defined in part 1.

type DataStore interface {
   Init() error
   GetItem(id string) (model.Item, error)
   GetItems() []model.Item
   AddItem(item model.Item) (model.Item, error)
   UpdateItem(item model.Item, id string) error
}

To get the application hooked up to a MongoDB instance, we require two things

1. A new MongoDB implementation of DataStore
2. Update the app to use the new implementation

Thats it – No other code changed are required as the database implementation has been abstracted from the rest of the application code. SOLID at its best (Single Responsibility, Open Close and Liskov Substitution principles all on display), making a change to the database (an implementation detail), a very localised change without the need for major surgery to the codebase.

A new go source file is created, mongodatastore.go with a new struct. To connect to MongoDB will need a URL to establish a connection, this wasn’t required for the InMemory, so as this is Mongo specific implementation, we can add the url as a parameter to the MongoDBDataStore struct. We also need a few more variables to hold the references to the MongoClient, collection and context, as well as the database and collection name, so adding these as properties to the struct makes sense.

type MongoDBDataStore struct {
   Url string
   DatabaseName string
   CollectionName string
   client *mongo.Client
   collection *mongo.Collection
   ctx context.Context
}

Now we can flesh out the implementation of the interface. Using documentation from an official MongoDB blog post as a guide, it’s fairly simple to get the CRUD operations (minus delete) implemented. Let start with Initialisation

func (d* MongoDBDataStore) Init() error {
   d.ctx = context.TODO()

   clientOptions := options.Client().ApplyURI(d.Url)
   client, err := mongo.Connect(d.ctx, clientOptions)
   if err != nil {
      log.Println(err)
   }

   err = client.Ping(d.ctx, nil)
   if err != nil {
      log.Println(err)
      return err
   }

   d.collection = client.Database(d.DatabaseName).Collection(d.CollectionName)
   return nil
}

Notice here we are using a pointer (d*) to the MongoDBDataStore. This struct now contains state, so we want to pass around the reference (the same instance of the struct in memory) not make copies of the struct for each call.

Get Item By ID

func (d* MongoDBDataStore) GetItem(id string) (model.Item, error) {
   // convert the ObjectID struct to a string representation (the ID)
   ID, err := primitive.ObjectIDFromHex(id)
   if err != nil {
      log.Println(err)
      return model.Item{}, err
   }

   singleResult := d.collection.FindOne(d.ctx, bson.M{"_id": ID})

   if singleResult == nil {
      log.Println("item not found for ID: " + id)
      return model.Item{}, errors.New("item not found for ID: " + id)
   }

   // Convert the JSON result to a struct
   var elem model.Item
   err = singleResult.Decode(&elem)
   if err != nil {
      log.Println(err)
      return model.Item{}, err
   }

   return elem, nil
}

Here we create a filter to search for a specific ID value, stored in _id which is a unique ID that MongoDB uses when creating documents. The result is then converted (marshalled) from JSON into the Go struct and returned by the function (passed back to the HTTP controller).

Get All Items

func (d *MongoDBDataStore) GetItems() []model.Item {
   var cur, err = d.collection.Find(d.ctx, bson.M{})
   var results []model.Item
   
   if err != nil {
      log.Println(err)
   } else {
      for cur.Next(d.ctx) {
         var elem model.Item
         err := cur.Decode(&elem)
         if err != nil {
            log.Println(err)
         }

         results = append(results, elem)
      }
      cur.Close(d.ctx)
   }

   return results
}

This time we use Find rather than FindOne. No filtering is applied, so we get all the results. In practice this isn’t ideal, as what if the database contained millions of rows? in this case we would want to page the results, but again for this demo all will be fine for now.

Add Item

func (d *MongoDBDataStore) AddItem(item model.Item) (model.Item, error) {
   insertResult, err := d.collection.InsertOne(d.ctx, item)
   if err != nil {
      log.Println(err)
      return item, err
   }

   id := insertResult.InsertedID
   item.Id = id.(primitive.ObjectID).Hex()
   return item, nil
}

Adding an item is pretty straightforward, the Item struct is passed to the InsertOne method, and if successful a new Item is inserted and assigned a new unique ID. The ID can be obtained from the returned struct. This is not a string, so the ID string needs to be extracted and converted before assigned to the item.ID and returned via the REST controller.

Update Item

func (d *MongoDBDataStore) UpdateItem(item model.Item, id string) error {
   ID, err := primitive.ObjectIDFromHex(id)
   if err != nil {
      log.Println(err)
      return err
   }

   _, err = d.collection.ReplaceOne(d.ctx, bson.M{"_id": ID}, item)
   if err != nil {
      log.Println(err)
      return err
   }
   
   return nil
}

There is an in-built MonogDB method called RepalceOne, which does what to says on the tin. It will replace the document, retaining its original ID.

Summary

This has been a fun learning experience, I have enjoyed diving into the Go programming language and implementing a simple service. The Go documentation is very good and although the language itself is fairly small, there are some really good supporting 3rd party libraries for many common tasks. I haven’t looked at these as yet, but one in particular Gorilla/Mux looks really interesting to simplify the process of creating a web service, although for a very simple service, this two-part blog shows it’s fairly easy to get up and running. As usual with this, making it production ready takes a lot more more time and effort (and testing!), but the purpose of this was a first look at the technology for learning purposes.

MongoDB is fantastic, it has great documentation and widely supported. Next steps will be to look at security, securing the Database and also the public REST API using JWT’s. I am loving working with MongoDB, so more to follow on this in future blogs.

In case you are interested, the full source code is here