Unmarshal CSV record into struct in Go

Learn unmarshal csv record into struct in go with practical examples, diagrams, and best practices. Covers csv, go, unmarshalling development techniques with visual explanations.
Tags:
Categories:

Unmarshal CSV Records into Go Structs with Ease

Hero image for Unmarshal CSV record into struct in Go

Learn how to efficiently parse CSV data into Go structs, handling common challenges like header mapping, type conversion, and error management for robust data processing.

Comma Separated Values (CSV) is a ubiquitous format for exchanging tabular data. In Go, processing CSV files often involves reading each record and then mapping its fields to a corresponding struct. This article will guide you through the process of unmarshalling CSV records into Go structs, covering essential techniques for robust and maintainable code.

Understanding the Challenge: CSV to Struct Mapping

When you read a CSV file, each line typically represents a record, and fields within that line are separated by a delimiter (commonly a comma). A Go struct, on the other hand, defines a fixed set of fields with specific types. The challenge lies in correctly associating CSV columns with struct fields, converting string values to appropriate Go types, and handling potential errors during this process.

flowchart TD
    A[Read CSV File] --> B{Read Record Line by Line}
    B --> C{Parse Fields}
    C --> D{Map Fields to Struct Properties}
    D --> E{Convert String to Target Type}
    E --> F[Populate Struct Instance]
    F --> G{Handle Errors?}
    G -->|Yes| H[Log Error/Skip Record]
    G -->|No| I[Add Struct to List]
    I --> B
    B -- EOF --> J[End Process]

Workflow for Unmarshalling CSV Records into Go Structs

Basic CSV Unmarshalling with encoding/csv

Go's standard library provides the encoding/csv package, which is excellent for reading and writing CSV data. While it doesn't directly support unmarshalling into structs like encoding/json, it provides the foundation. You'll typically read records as []string and then manually map them to your struct fields.

package main

import (
	"encoding/csv"
	"fmt"
	"io"
	"os"
	"strconv"
)

type Product struct {
	ID    int
	Name  string
	Price float64
	Stock int
}

func main() {
	csvData := `ID,Name,Price,Stock
1,Laptop,1200.50,100
2,Mouse,25.00,500
3,Keyboard,75.99,200`

	r := csv.NewReader(new(stringReader))
	r.Comma = ','

	// Read header
	header, err := r.Read()
	if err != nil {
		fmt.Println("Error reading header:", err)
		return
	}

	// Map header to struct fields (simple example, more robust needed for real apps)
	headerMap := make(map[string]int)
	for i, col := range header {
		headerMap[col] = i
	}

	var products []Product
	for {
		record, err := r.Read()
		if err == io.EOF {
			break
		}
		if err != nil {
			fmt.Println("Error reading record:", err)
			continue
		}

		// Basic validation: ensure record has enough fields
		if len(record) != len(header) {
			fmt.Printf("Skipping malformed record: %v\n", record)
			continue
		}

		var p Product
		p.ID, _ = strconv.Atoi(record[headerMap["ID"]])
		p.Name = record[headerMap["Name"]]
		p.Price, _ = strconv.ParseFloat(record[headerMap["Price"]], 64)
		p.Stock, _ = strconv.Atoi(record[headerMap["Stock"]])

		products = append(products, p)
	}

	fmt.Println("Parsed Products:")
	for _, p := range products {
		fmt.Printf("ID: %d, Name: %s, Price: %.2f, Stock: %d\n", p.ID, p.Name, p.Price, p.Stock)
	}
}

// stringReader is a helper to treat a string as an io.Reader
type stringReader struct {
	s *os.File
}

func (sr *stringReader) Read(p []byte) (n int, err error) {
	// For simplicity, this example uses a dummy stringReader. 
	// In a real scenario, you'd use `strings.NewReader(csvData)`.
	// This is just to make the example self-contained without file I/O.
	return 0, io.EOF 
}

// A more practical stringReader implementation:
// import "strings"
// r := csv.NewReader(strings.NewReader(csvData))

Manual CSV parsing and mapping to a struct using encoding/csv.

Leveraging Third-Party Libraries for Simplicity

Manually mapping fields and handling type conversions can become tedious and error-prone, especially with many columns or complex types. Several excellent third-party libraries simplify this process by using struct tags, similar to encoding/json.

One popular library is gocsv. It allows you to define struct tags to specify the corresponding CSV column name, making the unmarshalling process much cleaner and more robust.

package main

import (
	"fmt"
	"strings"

	"github.com/gocarina/gocsv"
)

type ProductWithTags struct {
	ID    int     `csv:"ID"`
	Name  string  `csv:"Name"`
	Price float64 `csv:"Price"`
	Stock int     `csv:"Stock"`
}

func main() {
	csvData := `ID,Name,Price,Stock
1,Laptop,1200.50,100
2,Mouse,25.00,500
3,Keyboard,75.99,200`

	var products []ProductWithTags

	err := gocsv.UnmarshalString(csvData, &products)
	if err != nil {
		fmt.Println("Error unmarshalling CSV:", err)
		return
	}

	fmt.Println("Parsed Products (using gocsv):")
	for _, p := range products {
		fmt.Printf("ID: %d, Name: %s, Price: %.2f, Stock: %d\n", p.ID, p.Name, p.Price, p.Stock)
	}

	// Example with a different delimiter (e.g., semicolon)
	semicolonCsvData := `ID;Name;Price;Stock\n4;Monitor;300.00;50\n5;Webcam;50.00;150`
	var moreProducts []ProductWithTags

	gocsv.Set  // This line is a placeholder for setting custom delimiter
	// To set a custom delimiter for gocsv, you'd typically create a custom CSV reader:
	// csvReader := csv.NewReader(strings.NewReader(semicolonCsvData))
	// csvReader.Comma = ';'
	// err = gocsv.UnmarshalCSV(csvReader, &moreProducts)
	// if err != nil {
	// 	fmt.Println("Error unmarshalling semicolon CSV:", err)
	// 	return
	// }
	// fmt.Println("\nParsed Products (semicolon delimited):")
	// for _, p := range moreProducts {
	// 	fmt.Printf("ID: %d, Name: %s, Price: %.2f, Stock: %d\n", p.ID, p.Name, p.Price, p.Stock)
	// }
}

Using gocsv to unmarshal CSV data into a struct with tags.