Writing Custom Terraform Providers in Go

Terraform has providers for AWS, Azure, GCP, Kubernetes, and hundreds of other services. But what happens when you need to manage resources in an internal API that no public provider supports? You write your own. Custom providers let you bring any API — your internal service catalog, a custom DNS system, or a configuration management platform — into the Terraform workflow. And the best part: the Terraform Plugin SDK v2 makes it surprisingly approachable.

Why Build a Custom Provider?

Public providers cover the major cloud platforms, but every organization has internal systems that Terraform cannot manage out of the box:

Use Case	Example	Why Custom Provider?
Internal service catalog	Register microservices with metadata	No public API provider exists
Custom DNS management	Internal BIND or PowerDNS API	Organization-specific endpoints
Config management API	Feature flags, application settings	Proprietary internal system
Certificate management	Internal PKI / Vault-like system	Custom issuance workflow
Access control	Internal RBAC or permission system	Unique business logic

A custom provider means your team can write terraform apply and provision internal resources alongside cloud infrastructure in a single plan.

Provider Architecture

Every Terraform provider follows the same architecture. Understanding this structure is the key to building one.

terraform-provider-example/
├── main.go                    # Entry point
├── provider.go                # Provider definition (schema + resources)
├── resource_service.go        # Resource CRUD implementation
├── data_source_service.go     # Data source read implementation
├── go.mod                     # Go module definition
└── go.sum                     # Dependency checksums

A provider has three core components:

1. Provider schema — configuration needed to authenticate and connect to your API (base URL, API keys, tokens).
2. Resources — represent objects you create, read, update, and delete (CRUD).
3. Data sources — read-only lookups for existing objects.

Setting Up the Go Project

Initialize a Go module and pull in the Terraform Plugin SDK:

mkdir terraform-provider-servicecatalog
cd terraform-provider-servicecatalog
go mod init github.com/your-org/terraform-provider-servicecatalog
go get github.com/hashicorp/terraform-plugin-sdk/v2

Your go.mod will look like this:

module github.com/your-org/terraform-provider-servicecatalog

go 1.21

require (
    github.com/hashicorp/terraform-plugin-sdk/v2 v2.33.0
)

Implementing the Provider

The provider definition tells Terraform what configuration it needs and which resources it offers.

// provider.go
package main

import (
    "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
)

func Provider() *schema.Provider {
    return &schema.Provider{
        Schema: map[string]*schema.Schema{
            "api_url": {
                Type:        schema.TypeString,
                Required:    true,
                DefaultFunc: schema.EnvDefaultFunc("SERVICECATALOG_API_URL", nil),
                Description: "Base URL of the Service Catalog API",
            },
            "api_token": {
                Type:        schema.TypeString,
                Required:    true,
                Sensitive:   true,
                DefaultFunc: schema.EnvDefaultFunc("SERVICECATALOG_API_TOKEN", nil),
                Description: "Authentication token for the API",
            },
        },
        ResourcesMap: map[string]*schema.Resource{
            "servicecatalog_service": resourceService(),
        },
        DataSourcesMap: map[string]*schema.Resource{
            "servicecatalog_service": dataSourceService(),
        },
        ConfigureContextFunc: providerConfigure,
    }
}

The ConfigureContextFunc creates an API client that every resource and data source can use:

// provider.go (continued)
package main

import (
    "context"
    "net/http"

    "github.com/hashicorp/terraform-plugin-sdk/v2/diag"
    "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
)

type APIClient struct {
    BaseURL    string
    Token      string
    HTTPClient *http.Client
}

func providerConfigure(ctx context.Context, d *schema.ResourceData) (interface{}, diag.Diagnostics) {
    apiURL := d.Get("api_url").(string)
    apiToken := d.Get("api_token").(string)

    client := &APIClient{
        BaseURL:    apiURL,
        Token:      apiToken,
        HTTPClient: &http.Client{},
    }

    return client, nil
}

Implementing a Resource with CRUD

Each resource needs four functions: Create, Read, Update, and Delete. Here is a servicecatalog_service resource:

// resource_service.go
package main

import (
    "context"
    "github.com/hashicorp/terraform-plugin-sdk/v2/diag"
    "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
    "github.com/hashicorp/terraform-plugin-sdk/v2/helper/validation"
)

func resourceService() *schema.Resource {
    return &schema.Resource{
        CreateContext: resourceServiceCreate,
        ReadContext:   resourceServiceRead,
        UpdateContext: resourceServiceUpdate,
        DeleteContext: resourceServiceDelete,
        Schema: map[string]*schema.Schema{
            "name": {
                Type:         schema.TypeString,
                Required:     true,
                ValidateFunc: validation.StringLenBetween(3, 64),
            },
            "owner_team": {
                Type:     schema.TypeString,
                Required: true,
            },
            "tier": {
                Type:         schema.TypeString,
                Optional:     true,
                Default:      "standard",
                ValidateFunc: validation.StringInSlice([]string{"critical", "standard", "best-effort"}, false),
            },
            "endpoint_url": {
                Type:         schema.TypeString,
                Optional:     true,
                ValidateFunc: validation.IsURLWithHTTPorHTTPS,
            },
            // Computed attributes (set by the API)
            "created_at": {
                Type:     schema.TypeString,
                Computed: true,
            },
        },
        Importer: &schema.ResourceImporter{
            StateContext: schema.ImportStatePassthroughContext,
        },
    }
}

func resourceServiceCreate(ctx context.Context, d *schema.ResourceData, m interface{}) diag.Diagnostics {
    client := m.(*APIClient)

    service := map[string]string{
        "name":       d.Get("name").(string),
        "owner_team": d.Get("owner_team").(string),
        "tier":       d.Get("tier").(string),
    }

    // POST to API, get back ID
    id, err := client.CreateService(service)
    if err != nil {
        return diag.FromErr(err)
    }

    d.SetId(id)
    return resourceServiceRead(ctx, d, m)
}

The Read, Update, and Delete functions follow the same pattern: get the client from m, call the API, and set values on d.

Implementing a Data Source

Data sources are simpler since they only need a Read function:

// data_source_service.go
package main

import (
    "context"
    "github.com/hashicorp/terraform-plugin-sdk/v2/diag"
    "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
)

func dataSourceService() *schema.Resource {
    return &schema.Resource{
        ReadContext: dataSourceServiceRead,
        Schema: map[string]*schema.Schema{
            "name": {
                Type:     schema.TypeString,
                Required: true,
            },
            "owner_team": {
                Type:     schema.TypeString,
                Computed: true,
            },
            "tier": {
                Type:     schema.TypeString,
                Computed: true,
            },
        },
    }
}

func dataSourceServiceRead(ctx context.Context, d *schema.ResourceData, m interface{}) diag.Diagnostics {
    client := m.(*APIClient)
    name := d.Get("name").(string)

    service, err := client.GetServiceByName(name)
    if err != nil {
        return diag.FromErr(err)
    }

    d.SetId(service.ID)
    d.Set("owner_team", service.OwnerTeam)
    d.Set("tier", service.Tier)

    return nil
}

The Entry Point

The main.go file is minimal. It just registers the provider:

// main.go
package main

import (
    "github.com/hashicorp/terraform-plugin-sdk/v2/plugin"
)

func main() {
    plugin.Serve(&plugin.ServeOpts{
        ProviderFunc: Provider,
    })
}

Schema Types and Validation

The SDK provides several schema types and built-in validators:

Schema Type	Go Type	Use Case
TypeString	string	Names, IDs, URLs
TypeInt	int	Counts, ports, sizes
TypeBool	bool	Feature flags, toggles
TypeFloat	float64	Percentages, thresholds
TypeList	[]interface{}	Ordered collections
TypeSet	*schema.Set	Unordered unique collections
TypeMap	map[string]interface{}	Key-value pairs

Validation functions keep bad input out of your API:

"port": {
    Type:         schema.TypeInt,
    Required:     true,
    ValidateFunc: validation.IntBetween(1, 65535),
},
"environment": {
    Type:         schema.TypeString,
    Required:     true,
    ValidateFunc: validation.StringInSlice([]string{"dev", "staging", "prod"}, false),
},
"cidr_block": {
    Type:         schema.TypeString,
    Required:     true,
    ValidateFunc: validation.IsCIDR,
},

Acceptance Testing

Terraform provides a testing framework that runs real plans and applies against your API:

// resource_service_test.go
package main

import (
    "testing"
    "github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource"
)

func TestAccServiceBasic(t *testing.T) {
    resource.Test(t, resource.TestCase{
        PreCheck:          func() { testAccPreCheck(t) },
        ProviderFactories: providerFactories,
        CheckDestroy:      testAccCheckServiceDestroy,
        Steps: []resource.TestStep{
            {
                Config: `
                    resource "servicecatalog_service" "test" {
                        name       = "test-service"
                        owner_team = "platform"
                        tier       = "standard"
                    }
                `,
                Check: resource.ComposeTestCheckFunc(
                    resource.TestCheckResourceAttr("servicecatalog_service.test", "name", "test-service"),
                    resource.TestCheckResourceAttr("servicecatalog_service.test", "tier", "standard"),
                    resource.TestCheckResourceAttrSet("servicecatalog_service.test", "created_at"),
                ),
            },
        },
    })
}

Run acceptance tests with:

TF_ACC=1 go test ./... -v -timeout 30m

Publishing to the Terraform Registry

To publish your provider to the public registry:

1. Name your repo terraform-provider-<name> on GitHub.
2. Tag a release following semver (e.g., v1.0.0).
3. Sign the release with a GPG key.
4. Register the GPG key with the Terraform Registry.
5. Sign in to registry.terraform.io and publish.

For internal use without the public registry, configure a ~/.terraformrc file:

provider_installation {
  dev_overrides {
    "your-org/servicecatalog" = "/path/to/go/bin"
  }
  direct {}
}

Provider Debugging

When things go wrong, attach a debugger to your provider process:

// main.go with debug support
func main() {
    var debugMode bool
    flag.BoolVar(&debugMode, "debug", false, "set to true to run the provider with support for debuggers")
    flag.Parse()

    opts := &plugin.ServeOpts{ProviderFunc: Provider}

    if debugMode {
        err := plugin.Debug(context.Background(), "your-org/servicecatalog", opts)
        if err != nil {
            log.Fatal(err.Error())
        }
        return
    }

    plugin.Serve(opts)
}

Run with dlv exec ./terraform-provider-servicecatalog -- -debug, then set the TF_REATTACH_PROVIDERS environment variable with the output to connect Terraform to your debugger session.

Closing Note

Building a custom Terraform provider is the bridge between "we manage cloud resources with Terraform" and "we manage everything with Terraform." The Plugin SDK v2 handles the hard parts — protocol negotiation, state management, plan diffing — so you can focus on the API integration logic. Start with a single resource, get the CRUD cycle working, write acceptance tests, and expand from there. Once your team sees internal resources appearing in terraform plan alongside their AWS and Azure infrastructure, there is no going back.