docs

Synkronus Server Reference

Complete technical reference for the Synkronus server component.

Want to get a server running quickly? See the Synkronus Quickstart guide for a simple Docker/Podman setup with automated TLS provisioning.

IT / production hosting? See Server Architecture for IT and Security reference.

Overview

Synkronus is a robust synchronization API server built with Go. It provides RESTful endpoints for data synchronization, app bundle management, attachment handling, user management, and form specifications. The server uses PostgreSQL for data storage and JWT for authentication.

Released container images

Production deployments should pin a release tag rather than :latest:

ghcr.io/opendataensemble/synkronus:v1.1.1

Images are published on GitHub Container Registry for each ODE release.

Architecture

Technology Stack

Project Structure

synkronus/
├── cmd/synkronus/         # Application entry point
├── internal/              # Private application code
│   ├── api/               # API definition and OpenAPI integration
│   ├── handlers/          # HTTP request handlers
│   ├── models/            # Domain models
│   ├── repository/        # Data access layer
│   └── services/          # Business logic
└── pkg/                   # Public libraries
    ├── auth/              # Authentication utilities
    ├── database/          # Database connection and migrations
    ├── logger/            # Structured logging
    ├── middleware/        # HTTP middleware
    └── openapi/           # OpenAPI generated code

Configuration

Required Environment Variables

Variable Description Example
JWT_SECRET Secret key for JWT signing (generate with openssl rand -base64 32) AbCd1234=
DB_CONNECTION PostgreSQL connection string postgres://user:pass@localhost:5432/synkronus

Optional Environment Variables

Variable Default Description
PORT 8080 HTTP server port
LOG_LEVEL info Logging level: debug, info, warn, error
MAX_VERSIONS_KEPT 5 Number of app bundle versions to retain
ADMIN_USERNAME admin Initial admin username
ADMIN_PASSWORD admin Initial admin password (change after first login!)

Database Configuration

PostgreSQL Requirements:

Example connection string:

postgres://synkronus_user:[email protected]:5432/synkronus?sslmode=require

Parameters:

File Storage Configuration

The server stores files at <data_root>/app-bundle/ and <data_root>/attachments/:

/app/data/                           # Data root (from binary location)
├── app-bundle/
│   ├── active/                      # Active app bundle
│   │   ├── forms/
│   │   │   ├── household.json
│   │   │   └── hh_person.json
│   │   ├── question_types/
│   │   └── metadata.json
│   └── versions/                    # Historical versions
│       ├── 1.0.0/
│       ├── 0.9.0/
│       └── ...
└── attachments/                     # Observation attachments (photos, files)
    ├── obs-123-photo.jpg
    ├── obs-456-audio.m4a
    └── ...

Docker Volume Mount:

volumes:
  - synkronus_data:/app/data  # Single volume containing all mutable data

Directory Permissions (Docker):

Core Features

Data Synchronization

App Bundle Management

Attachment Handling

User Management

Form Specifications

API Endpoints

Authentication

POST /auth/login

Authenticate user and receive JWT token.

Request:

{
  "username": "user",
  "password": "password"
}

Response:

{
  "token": "eyJhbGciOiJIUzI1NiIs...",
  "refreshToken": "eyJhbGciOiJIUzI1NiIs...",
  "expiresIn": 3600
}

POST /auth/refresh

Refresh expired JWT token.

Request:

{
  "refreshToken": "eyJhbGciOiJIUzI1NiIs..."
}

Synchronization

POST /sync/pull

Pull changes from server.

Request:

{
  "clientId": "client-123",
  "currentVersion": 100,
  "schemaTypes": ["survey", "visit"]
}

Response:

{
  "changes": {
    "observations": [...]
  },
  "timestamp": 150
}

POST /sync/push

Push changes to server.

Request:

{
  "clientId": "client-123",
  "changes": {
    "observations": [...]
  }
}

Response:

{
  "timestamp": 150,
  "conflicts": []
}

App Bundles

GET /app-bundle/manifest

Get current app bundle manifest.

Response:

{
  "version": "20250114-123456",
  "files": [...],
  "hash": "abc123..."
}

GET /app-bundle/download/{path}

Download app bundle file.

Path Parameters:

POST /app-bundle/push

Upload new app bundle (admin only).

Request: Multipart form with bundle file

Response:

{
  "version": "20250114-123456",
  "manifest": {...}
}

GET /app-bundle/versions

List all app bundle versions.

POST /app-bundle/switch

Switch active bundle version (admin only).

Attachments

GET /attachments/manifest

Get attachment manifest.

Query Parameters:

GET /attachments/{id}

Download attachment file.

POST /attachments

Upload attachment (multipart form).

Form Specifications

GET /formspecs/{formType}/{version}

Get form specification.

Path Parameters:

POST /formspecs

Create form specification (admin only).

Users

GET /users

List all users (admin only).

POST /users/create

Create new user (admin only).

GET /users/{username}

Get user details.

PUT /users/{username}

Update user (admin only).

DELETE /users/{username}

Delete user (admin only).

Data Export

GET /data/export

Export observations as Parquet ZIP.

Query Parameters:

Configuration

Environment Variables

Variable Description Default Required
PORT HTTP server port 8080 No
DB_CONNECTION PostgreSQL connection string - Yes
JWT_SECRET Secret for JWT signing - Yes
LOG_LEVEL Logging level (debug, info, warn, error) info No
APP_BUNDLE_PATH Directory for app bundles ./data/app-bundles No
MAX_VERSIONS_KEPT Maximum bundle versions to keep 5 No
ADMIN_USERNAME Initial admin username admin No
ADMIN_PASSWORD Initial admin password admin No

Example Configuration

PORT=8080
DB_CONNECTION=postgres://user:password@localhost:5432/synkronus?sslmode=disable
JWT_SECRET=your-secret-key-change-this-in-production
LOG_LEVEL=info
APP_BUNDLE_PATH=./data/app-bundles
MAX_VERSIONS_KEPT=5
ADMIN_USERNAME=admin
ADMIN_PASSWORD=admin

Database Schema

Observations Table

Column Type Description
id UUID Primary key
form_type VARCHAR Form type identifier
data JSONB Observation data
created_at TIMESTAMP Creation timestamp
updated_at TIMESTAMP Last update timestamp
deleted BOOLEAN Soft delete flag
version INTEGER Version number (auto-increment)

Users Table

Column Type Description
id UUID Primary key
username VARCHAR Unique username
password_hash VARCHAR Hashed password
role VARCHAR User role (read-only, read-write, admin)
created_at TIMESTAMP Creation timestamp

App Bundle Versions Table

Column Type Description
version VARCHAR Version identifier
is_active BOOLEAN Active version flag
created_at TIMESTAMP Creation timestamp

Synchronization Protocol

Two-Phase Sync

Phase 1: Observation Sync

  1. Client requests changes via /sync/pull
  2. Server returns observations changed since client’s version
  3. Client applies changes locally
  4. Client pushes local changes via /sync/push
  5. Server applies changes and returns new version

Phase 2: Attachment Sync

  1. Client requests attachment manifest
  2. Server returns list of attachments to download
  3. Client downloads missing attachments
  4. Client uploads pending attachments
  5. Server confirms receipt

Conflict Resolution

Conflicts are detected when:

Resolution strategy:

Security

Authentication

Authorization

Data Protection

Deployment

Docker Deployment

See Deployment Guide for complete deployment instructions.

Quick Start

docker compose up -d

Production Setup

  1. Configure environment variables
  2. Set up PostgreSQL database
  3. Configure reverse proxy (Nginx)
  4. Set up SSL/TLS certificates
  5. Configure monitoring and logging

Monitoring

Health Check

curl http://localhost:8080/health

Returns OK if server is healthy.

Logging

Structured logging with levels:

Metrics

Key metrics to monitor:

Performance

Optimization Strategies

Scaling

Troubleshooting

Common Issues

Database Connection Errors:

Authentication Failures:

Sync Failures:

API Versioning

The API supports versioning via the x-api-version header:

x-api-version: 1.0.0

Version negotiation allows clients to request specific API versions.