Microservices in Golang, pt3 - Serialize data

We’ll now check out to Serialize data with encoding/json with help of the package encoding/json - the documentation can be found here .

The goals is going to be converting our Data struct into the JSON representation, for that we’ll be using a Songs structure.

In our data package Songs, we’ll add a method to the Song as our data access model - thus avoiding exposing the database connection or writing into the database in the handler, etc. By abstracting the detailed logic of where the data is coming from, from the rest of the code, we have it separately in our data access model.

package data

import "time"

// Song defines the structure for an song API
type Song struct {
	ID        int
	Band      string
	Title     string
	Price     float32
	SKU       string
	CreatedOn string
	UpdatedOn string
	DeletedOn string
}

var songList = []*Song{
	&Song{
		ID:        1,
		Band:      "The Rockers",
		Title:     "Train to rainbow piece",
		Price:     9.99,
		SKU:       "SONG001",
		CreatedOn: time.Now().UTC().String(),
		UpdatedOn: time.Now().UTC().String(),
	},
	&Song{
		ID:        2,
		Band:      "Pickle burger pinks",
		Title:     "Tomato salad",
		Price:     4.99,
		SKU:       "SONG002",
		CreatedOn: time.Now().UTC().String(),
		UpdatedOn: time.Now().UTC().String(),
	},
	&Song{
		ID:        3,
		Band:      "Triangular planet",
		Title:     "Feets go only so deep",
		Price:     6.50,
		SKU:       "SONG003",
		CreatedOn: time.Now().UTC().String(),
		UpdatedOn: time.Now().UTC().String(),
	},
}

The function GetSongs and is going to return all of my Songs and is a quite simple function because we are returning a statically defined list, for this example.

func GetSongs() []*Song {
	return songList
}

At this point you should know the pattern and have a handler for our Songs, where we get a list of Songs.

package handlers

import (
	"log"
	"net/http"

	"example.com/go-intro-microservices-pt2/data"
)

type Songs struct {
	l *log.Logger
}

func NewSongs(l *log.Logger) *Songs {
	return &Songs{l}
}

func (p *Songs) ServeHTTP(rw http.ResponseWriter, r http.Request) {
	lp := data.GetSongs()
}

We now need to return the list of products to the user, so we need to convert our lp that is a struct into json.

The simplest method we can find is json.Marshal, that the signature is:

Marshal func(v interface{}) ([]byte, error)

Marshal returns the JSON encoding of v.

The Marshal returns a slice of data and an error. Since the http.ResponseWrite signature has a Write method, we use that to responde to the user.

func (p *Songs) ServeHTTP(rw http.ResponseWriter, r *http.Request) {
	lp := data.GetSongs()
	d, err := json.Marshal(lp)
	if err != nil {
		http.Error(rw, "Unable to marshal json of songs!", http.StatusInternalServerError)
	}
	rw.Write(d)
}

A call to the service using curl -v localhost:9000/api/v1/songs would return:

* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 9000 (#0)
> GET /api/v1/songs HTTP/1.1
> Host: localhost:9000
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Sun, 1 Jan 1970 00:00:00 GMT
< Content-Length: 630
< Content-Type: text/plain; charset=utf-8
<
{ [630 bytes data]
100   630  100   630    0     0   205k      0 --:--:-- --:--:-- --:--:--  205k
* Connection #0 to host localhost left intact
* Closing connection 0
[
  {
    "ID": 1,
    "Band": "The Rockers",
    "Title": "Train to rainbow piece",
    "Price": 9.99,
    "SKU": "SONG001",
    "CreatedOn": "1970-1-1 00:00:00.807375 +0000 UTC",
    "UpdatedOn": "1970-1-1 00:00:00.807375 +0000 UTC",
    "DeletedOn": ""
  },
  {
    "ID": 2,
    "Band": "Pickle burger pinks",
    "Title": "Tomato salad",
    "Price": 4.99,
    "SKU": "SONG002",
    "CreatedOn": "1970-1-1 00:00:00.807375 +0000 UTC",
    "UpdatedOn": "1970-1-1 00:00:00.807375 +0000 UTC",
    "DeletedOn": ""
  },
  {
    "ID": 3,
    "Band": "Triangular planet",
    "Title": "Feets go only so deep",
    "Price": 6.5,
    "SKU": "SONG003",
    "CreatedOn": "1970-1-1 00:00:00.807375 +0000 UTC",
    "UpdatedOn": "1970-1-1 00:00:00.807375 +0000 UTC",
    "DeletedOn": ""
  }
]

Your response won’t look as pretty format, as I use jq and pipe the response of curl to jq.

curl -v localhost:9000/api/v1/songs | jq

There are cases where the struct fields or the data source have name cases that are not a good fit for your project, you can rename these quite easily if you want to transform them into camelcase, for example.

The json package provides a way to annotate fields, called field tags or struct tags. We can rename the output fields of our struct or even omit the fields we don’t want. You can find more about struct tags in the following digitalocean article called how to use struct tags in go .

type Song struct {
	ID        int     `json:"id"`
	Band      string  `json:"band"`
	Title     string  `json:"title"`
	Price     float32 `json:"price"`
	SKU       string  `json:"sku"`
	CreatedOn string  `json:"-"`
	UpdatedOn string  `json:"-"`
	DeletedOn string  `json:"-"`
}

Remember to restart your server before executing the curl command. You should get a different result based on your changes, lowercase id, band, title, price, sku and omitted the createdOn, updatedOn, DeletedOn:

[
  {
    "id": 1,
    "band": "The Rockers",
    "title": "Train to rainbow piece",
    "price": 9.99,
    "sku": "SONG001"
  },
  {
    "id": 2,
    "band": "Pickle burger pinks",
    "title": "Tomato salad",
    "price": 4.99,
    "sku": "SONG002"
  },
  {
    "id": 3,
    "band": "Triangular planet",
    "title": "Feets go only so deep",
    "price": 6.5,
    "sku": "SONG003"
  }
]

In the example above we used bytes as intermediates between the data and JSON representation on standard out. We can also stream JSON encodings directly to os.Writers like os.Stdout or even HTTP response bodies. Other example can be found here .

Our data/songs.go:

package data

import (
	"encoding/json"
	"io"
	"time"
)

// Song defines the structure for an song API
type Song struct {
	ID        int     `json:"id"`
	Band      string  `json:"band"`
	Title     string  `json:"title"`
	Price     float32 `json:"price"`
	SKU       string  `json:"sku"`
	CreatedOn string  `json:"-"`
	UpdatedOn string  `json:"-"`
	DeletedOn string  `json:"-"`
}

type Songs []*Song

func (s *Songs) ToJSON(w io.Writer) error {
	enc := json.NewEncoder(w)
	return enc.Encode(s)
}

func GetSongs() Songs {
	return songList
}

var songList = []*Song{
	&Song{
		ID:        1,
		Band:      "The Rockers",
		Title:     "Train to rainbow piece",
		Price:     9.99,
		SKU:       "SONG001",
		CreatedOn: time.Now().UTC().String(),
		UpdatedOn: time.Now().UTC().String(),
	},
	&Song{
		ID:        2,
		Band:      "Pickle burger pinks",
		Title:     "Tomato salad",
		Price:     4.99,
		SKU:       "SONG002",
		CreatedOn: time.Now().UTC().String(),
		UpdatedOn: time.Now().UTC().String(),
	},
	&Song{
		ID:        3,
		Band:      "Triangular planet",
		Title:     "Feets go only so deep",
		Price:     6.50,
		SKU:       "SONG003",
		CreatedOn: time.Now().UTC().String(),
		UpdatedOn: time.Now().UTC().String(),
	},
}

The handlers/songs.go:

package handlers

import (
	"log"
	"net/http"

	"example.com/go-intro-microservices-pt2/data"
)

type Songs struct {
	l *log.Logger
}

func NewSongs(l *log.Logger) *Songs {
	return &Songs{l}
}

func (p *Songs) ServeHTTP(rw http.ResponseWriter, r *http.Request) {
	ls := data.GetSongs()
	err := ls.ToJSON(rw)
	if err != nil {
		http.Error(rw, "Unable to marshal json of songs!", http.StatusInternalServerError)
	}
}

Finally, we need to refactor and make this the same format suitable to assign to specific HTTP Method in our Gorilla Mux, as we did with the Hello functions for each corresponding HTTP method, GET, PUT, DELETE, etc.

package handlers

import (
	"log"
	"net/http"

	"example.com/go-intro-microservices-pt2/data"
)

type Songs struct {
	l *log.Logger
}

func NewSongs(l *log.Logger) *Songs {
	return &Songs{l}
}

func (s *Songs) Get(rw http.ResponseWriter, r *http.Request) {
	ls := data.GetSongs()
	err := ls.ToJSON(rw)
	if err != nil {
		http.Error(rw, "Unable to marshal json of songs!", http.StatusInternalServerError)
	}
}

Update the main.go:

package main

import (
	"context"
	"log"
	"net/http"
	"os"
	"os/signal"
	"time"

	"example.com/go-intro-microservices-pt2/handlers"
	"github.com/gorilla/mux"
)

func main() {
	l := log.New(os.Stdout, "rest-api", log.LstdFlags)

	hh := handlers.NewHello(l)
	sh := handlers.NewSongs(l)

	sm := mux.NewRouter()

	api := sm.PathPrefix("/api/v1").Subrouter()
	api.HandleFunc("", hh.Get).Methods(http.MethodGet)
	api.HandleFunc("", hh.Post).Methods(http.MethodPost)
	api.HandleFunc("", hh.Put).Methods(http.MethodPut)
	api.HandleFunc("", hh.Delete).Methods(http.MethodDelete)
	api.HandleFunc("/songs", sh.Get).Methods(http.MethodGet)
	api.HandleFunc("", hh.NotFound)

	s := &http.Server{
		Addr:         ":9000",
		Handler:      sm,
		IdleTimeout:  120 * time.Second,
		ReadTimeout:  1 * time.Second,
		WriteTimeout: 1 * time.Second,
	}

	go func() {
		err := s.ListenAndServe()
		if err != nil {
			l.Fatal(err)
		}
	}()

	sigChan := make(chan os.Signal)
	signal.Notify(sigChan, os.Interrupt)
	signal.Notify(sigChan, os.Kill)

	sig := <-sigChan
	l.Println("Terminate received, gracefully shuttingdown...", sig)

	tc, cancel := context.WithTimeout(context.Background(), 30*time.Second)
	defer cancel()

	s.Shutdown(tc)
}

This can get quite verbose, so we could instead handle the routing in the original method definition of our songs struct if you have that preference and keep the main cleaner and use .Handle

func (p *Songs) ServeHTTP(rw http.ResponseWriter, r *http.Request) {
	if r.Method == http.MethodGet {
		p.getSongs(rw, r)
		return
	}

	// catch all
	rw.WriteHeader(http.StatusMethodNotAllowed)
}

Now, for HTTP POST, PUT, DELETE we need to compute data. So, similarly to what we’ve done with .ToJSON we’ll create .FromJSON in our data access model in data/songs

func (s *Song) FromJSON(r io.Reader) error {
	e := json.NewDecoder(r)
	return e.Decode(s)
}

Include a new method declared in our Songs handler.

func (s *Songs) Post(rw http.ResponseWriter, r *http.Request) {
	s.l.Println("Handle POST Songs")

	song := &data.Song{}

	err := song.FromJSON(r.Body)
	if err != nil {
		http.Error(rw, "Unable to unmarshal json of song", http.StatusBadRequest)
	}

	data.AddSong(song)
}

To conclude, register the path, function and method into our Mux.

api.HandleFunc("/songs", sh.Post).Methods(http.MethodPost)

After we re-start the server, we can POST data to the endpoint. We can omit the -X POST, but I’ve kept it here for clarity.

curl -v -X POST -d '{"band": "Paper Clips", "title": "Our favourite sunday", "price": 4.75, "sku": "xz"}'  localhost:9000/api/v1/songs

And finally, HTTP GET

curl -v localhost:9000/api/v1/songs | jq

To see

[
  {
    "id": 1,
    "band": "The Rockers",
    "title": "Train to rainbow piece",
    "price": 9.99,
    "sku": "SONG001"
  },
  {
    "id": 2,
    "band": "Pickle burger pinks",
    "title": "Tomato salad",
    "price": 4.99,
    "sku": "SONG002"
  },
  {
    "id": 3,
    "band": "Triangular planet",
    "title": "Feets go only so deep",
    "price": 6.5,
    "sku": "SONG003"
  },
  {
    "id": 4,
    "band": "Paper Clips",
    "title": "Our favourite sunday",
    "price": 4.75,
    "sku": "xz"
  }
]

Have in mind that after you HTTP POST, do not restart the server, because it’s stateless, we don’t persist our data in a database, as we have statically typed the data as an example only.

As I’ve wrote before, we are using Gorilla Mux to take advantage to built in functionality, such as path variables. When we HTTP PUT, we are sending a variable to indentify a particular item we want to update.

In our handlers/songs.go

func (s *Songs) Update(rw http.ResponseWriter, r *http.Request) {
	vars := mux.Vars(r)
	id, err := strconv.Atoi(vars["id"])
	if err != nil {
		http.Error(rw, "Unable to convert id", http.StatusBadRequest)
		return
	}

	s.l.Println("Handle PUT Songs, update song id", id)

	song := &data.Song{}

	err = song.FromJSON(r.Body)
	if err != nil {
		http.Error(rw, "Unable to unmarshal json of song", http.StatusBadRequest)
	}

	err = data.UpdateSong(id, song)
	if err == data.ErrSongNotFound {
		http.Error(rw, "Song not found", http.StatusNotFound)
		return
	}

	if err != nil {
		http.Error(rw, "Song not found", http.StatusInternalServerError)
		return
	}
}

In our data/songs.go

func UpdateSong(id int, s *Song) error {
	_, pos, err := findSong(id)
	if err != nil {
		return err
	}

	s.ID = id
	songList[pos] = s

	return nil
}

var ErrSongNotFound = fmt.Errorf("Song not found!")

func findSong(id int) (*Song, int, error) {
	for i, s := range songList {
		if s.ID == id {
			return s, i, nil
		}
	}

	return nil, -1, ErrSongNotFound
}

And finally in our main.go

api.HandleFunc("/songs/{id:[0-9]+}", sh.Update).Methods(http.MethodPut)

The HTTP PUT request through curl to update the song number 3:

curl -v -X PUT -d '{"band": "Zipzags", "title": "Songalicious", "price": 1.75, "sku": "abz1"}'  localhost:9000/api/v1/songs/3 | jq
comments powered by Disqus