Decisions Made

Question	Decision
Plugin name	shep-plugin-inventory
Capability	Single inventory capability
Var values	String-only for now (JSON-encoded for complex types)
Formats	YAML only (INI later)
YAML library	github.com/yaml/go-yaml (successor to unmaintained gopkg.in/yaml.v3)

---

Architecture Rationale

Looking at the architecture docs (shep-orchestration.md, shep-plugin-modeling.md), the pattern is clear:

1. Capabilities are verbs (what can be done): "parse", "execute", "schedule"
2. Domain types are nouns (what data flows): Inventory, Task, Host
3. Plugins implement capabilities, not domain types

The docs show this example flow:

playbook.yml ──► parse (plugin) ──► Inventory + TaskList

And this pattern for capability naming:

capability="parse"    → payload is ParseRequest or ParseResponse
capability="inventory.file" → inventory from file
capability="inventory.aws"  → inventory from AWS

Why shep-plugin-inventory (not shep-plugin-parser):

• The docs show parse capability returning Inventory + TaskList together for playbooks, but inventory and playbook parsing are semantically different
• Different inventory sources (file, AWS, GCP) would be different capabilities on potentially different plugins
• A generic "parser" conflates concerns: parsing playbooks vs parsing inventory vs parsing vars files

---

Prerequisite: Do We Need vars.json First?

No, we can start without it.

The vars.json schema defines valid variable names (avoiding reserved words). For inventory parsing, we need to handle host vars and group vars, but:

1. We can parse vars as opaque key-value pairs initially
2. Validation of var names can be added later
3. The inventory structure itself doesn't depend on vars schema

Approach: Parse vars as [Var] where Var is {key:string, value:string}. Complex values are JSON-encoded strings. Defer typed values and validation to a later phase.

---

Implementation Plan

Phase 1: Domain Schema Definition

Goal: Define FlatBuffers schemas for inventory domain types.

Files to create:

flatbuffers/
├── domain/
│   └── inventory.fbs      # Inventory, Host, Group, Var
└── capabilities/
    └── inventory.fbs      # InventoryRequest, InventoryResponse

Domain types (flatbuffers/domain/inventory.fbs):

namespace shep.domain;

// A variable (key-value pair)
// TODO: Add typed Value union (string | int | bool | list | map)
// For now, complex values are JSON-encoded strings
table Var {
  key:string;
  value:string;
}

// A host in the inventory
table Host {
  name:string;           // hostname or alias
  vars:[Var];            // host-specific variables
}

// A group of hosts
table Group {
  name:string;           // group name
  hosts:[string];        // host names (references, not embedded)
  children:[string];     // child group names
  vars:[Var];            // group variables
}

// Complete inventory
table Inventory {
  hosts:[Host];
  groups:[Group];
}

Capability types (flatbuffers/capabilities/inventory.fbs):

namespace shep.cap.inventory;

include "domain/inventory.fbs";

// Request to load inventory
table InventoryRequest {
  file_path:string;      // path to inventory file
  content:[ubyte];       // or raw content (for testing/piping)
}

// Response with parsed inventory
table InventoryResponse {
  inventory:shep.domain.Inventory;
  warnings:[string];     // non-fatal parse warnings
  error:string;          // fatal error message (if failed)
}

Update Makefile:

• Add rules to compile domain/ and capabilities/ schemas
• Update gen-schemas target

---

Phase 2: Plugin Implementation

Goal: Create shep-plugin-inventory that implements the inventory capability.

Files to create:

plugins/inventory/
├── main.go              # Plugin entry point, protocol handling
├── parser.go            # YAML parsing logic
└── parser_test.go       # Unit tests for parsing

Plugin structure (main.go):

1. Send handshake declaring inventory capability (version 1)
2. Wait for handshake response
3. Enter message loop:
   • Receive envelope with capability="inventory"
   • Deserialize InventoryRequest from payload
   • Parse inventory file/content
   • Build InventoryResponse with Inventory or error
   • Send response envelope
4. Handle shutdown message

Parser logic (parser.go):

1. YAML parsing:

   • Use github.com/yaml/go-yaml
   • Handle Ansible inventory structure:
     • Top-level keys are group names
     • all and ungrouped are special groups
     • Groups have hosts, children, vars keys
     • Hosts can be strings or objects with vars

2. Structure normalization:

   • Flatten nested host references
   • Resolve group membership
   • Collect all unique hosts
   • Build Inventory FlatBuffer

3. Error handling:

   • Return clear error messages in response
   • Collect warnings for non-fatal issues

---

Phase 3: Host Integration

Goal: Add inventory loading to the shep host.

Changes to cmd/shep/main.go:

1. Add inventory command group:

   shep inventory ls --inventory=hosts.yml
   shep i ls --inventory=hosts.yml  (short alias)

1. Add --inventory / -i flag for inventory file path

2. Verbose mode output:

   • List all hosts with addresses
   • List all groups with membership
   • Show variable inheritance (if verbose)

Changes to internal/host/plugin.go:

1. Add typed wrapper for inventory capability:

   func (m *Manager) LoadInventory(filePath string) (*Inventory, error)

1. Handle capability discovery for inventory

---

Phase 4: CLI Implementation

Goal: Implement shep inventory ls command.

Output format (default):

HOSTS:
  green.example.com     191.168.100.32
  blue.example.com      -
  192.168.100.1         -
  192.168.100.10        -

GROUPS:
  ungrouped (4 hosts)

Output format (verbose):

HOSTS:
  green.example.com
    address: 191.168.100.32
    vars:
      ansible_ssh_host: 191.168.100.32
      anyvariable: value
  blue.example.com
    address: blue.example.com
  ...

GROUPS:
  usa
    children: southeast, northeast, northwest, southwest
  southeast
    children: atlanta, raleigh
    vars:
      some_server: foo.southeast.example.com
      halon_system_timeout: 30
  atlanta
    hosts: host1, host2
  ...

Flags:

• --inventory, -i — path to inventory file (required)
• --format — output format: text (default), json, yaml
• --verbose, -v — show vars and full details

---

Phase 5: Testing

Unit tests (plugins/inventory/parser_test.go):

• Parse simple ungrouped inventory
• Parse hierarchical groups with children
• Parse host variables
• Parse group variables
• Handle empty groups
• Handle malformed YAML (error case)

Integration tests (internal/host/inventory_test.go):

• Start inventory plugin
• Load test inventory files
• Verify parsed structure matches expected
• Test error handling for missing files

Test data:

• Use existing testdata/inventory/ files
• Add edge case files as needed

---

File Checklist

New Files

flatbuffers/domain/inventory.fbs           # Domain types
flatbuffers/capabilities/inventory.fbs     # Capability request/response
plugins/inventory/main.go                  # Plugin entry point
plugins/inventory/parser.go                # Parsing logic
plugins/inventory/parser_test.go           # Unit tests
internal/host/inventory.go                 # Host-side typed wrapper (optional)
internal/host/inventory_test.go            # Integration tests

Modified Files

Makefile                                   # Add schema compilation rules
cmd/shep/main.go                           # Add inventory command

Generated Files (by flatc)

internal/fbs/shep/domain/Inventory.go
internal/fbs/shep/domain/Host.go
internal/fbs/shep/domain/Group.go
internal/fbs/shep/domain/Var.go
internal/fbs/shep/cap/inventory/InventoryRequest.go
internal/fbs/shep/cap/inventory/InventoryResponse.go

---

Dependencies

Go packages needed:

• github.com/yaml/go-yaml — YAML parsing (successor to gopkg.in/yaml.v3)

No new FlatBuffer dependencies (already using google/flatbuffers)

---

Milestones

1. [x] M1: Schema definition — FlatBuffers schemas compile, Go code generates
2. [x] M2: Plugin skeleton — Plugin starts, handshakes, handles capability
3. [x] M3: YAML parsing — Can parse test inventory files
4. [x] M4: Host integration — shep inventory ls works with basic output
5. [x] M5: Verbose mode — Full details with vars shown
6. [x] M6: Tests passing — Unit and integration tests complete

---

Future Work (Out of Scope)

• [ ] INI inventory format support
• [ ] Typed Value union for vars (string | int | bool | list | map)
• [ ] Variable name validation against vars.json schema
• [ ] Dynamic inventory (AWS, GCP, etc.) — separate plugins
• [ ] Playbook parsing — separate parse capability
• [ ] Variable inheritance resolution — host responsibility
• [ ] Host pattern matching — host responsibility