feat: add KV unmarshal feature with struct tag support #24
Conversation
Add Unmarshal() method to KVScanner that unmarshals KV entries into
structs using struct tags, similar to encoding/json. This provides a
convenient API for extracting structured data from SPOE messages.
Features:
- Struct tag support: `spoe:"keyname"` maps KV keys to struct fields
- Type support: string, []byte, int32, int64, uint32, uint64, bool, netip.Addr
- Optional fields via pointer types (set to nil if key not found)
- Zero-allocation key matching using NameEquals()
- Optimized reflection usage with cached field values and kinds
- Comprehensive test coverage
Performance:
The unmarshal feature trades some performance for convenience. Benchmarks
show manual iteration is ~3x faster, but unmarshal provides better
developer experience for non-hot paths.
Benchmark Results:
```
goos: linux
goarch: amd64
pkg: github.com/dropmorepackets/haproxy-go/pkg/encoding
cpu: 12th Gen Intel(R) Core(TM) i7-12700H
BenchmarkUnmarshal-20 3883784 334.2 ns/op 614 B/op 8 allocs/op
BenchmarkManualIteration-20 10422141 114.2 ns/op 157 B/op 5 allocs/op
```
Manual iteration: ~114 ns/op, 157 B/op, 5 allocs/op
Unmarshal: ~334 ns/op, 614 B/op, 8 allocs/op
The additional allocations come from:
- Field info slice setup (one-time per unmarshal call)
- Pointer field tracking map
- Reflection overhead for type checking
Usage Example:
```go
type RequestData struct {
Headers []byte `spoe:"headers"`
Status int32 `spoe:"status-code"`
IP netip.Addr `spoe:"client-ip"`
Optional *string `spoe:"optional-field"`
}
var data RequestData
if err := m.KV.Unmarshal(&data); err != nil {
// handle error
}
```
|
This does in fact align with the goals 😄 I just didn't had time/usage for this feature yet. My Idea was to implement additional abstractions in separate packages that are simpler but potentially slower. I would not use pointers but keep the default value and use |
Great! the reason I said "not aligned" was because I saw an emphasis on "reduce allocations" and I didnt know if you wanted to force the iterator pattern to be the only way cause that is obviously the most performant way to do this. Ye also let me know if you wish the function to be on the |
|
I am not sure if having it on the kvscanner is bad, but I would also not mind having it as functions in the encoding package. I guess we could do the same as the json package and do a Unmarshal + NewDecoder/NewEncoder for reusing things.
Thank you for implementing this 😄 |
2 participants
Hey feel free to close as unplanned if you feel this does not align to the project goal.
the
tldris, sometimes you may have a spoe message which is not called very often and technical debt to write a whole iterator pattern simply for this message, having the struct drive the datatype and the message keys add a convenient way to get the data.Add
Unmarshal()method to KVScanner that unmarshals KV entries into structs using struct tags, similar to encoding/json. This provides a convenient API for extracting structured data from SPOE messages.Features:
spoe:"keyname"maps KV keys to struct fieldsPerformance:
The unmarshal feature trades some performance for convenience. Benchmarks show manual iteration is ~3x faster, but unmarshal provides better developer experience for non-hot paths.
Benchmark Results:
The additional allocations come from:
Usage Example: