Documentation
¶
Index ¶
- Variables
- func ExtractRecords(streamFunc func(*models.StreamConfig) error)
- func GenerateSchema(record interface{}) (map[string]interface{}, error)
- func RecordMessage(record map[string]interface{}) error
- func UpdateSchema(existingSchema, newSchema map[string]interface{}) (map[string]interface{}, error)
- type TransformationMetrics
Constants ¶
This section is empty.
Variables ¶
View Source
var ExtractedChan = make(chan map[string]interface{})
View Source
var ProcessingWG sync.WaitGroup
View Source
var ResultChan = make(chan map[string]interface{}, 100)
View Source
var TransformMetrics = &TransformationMetrics{}
Functions ¶
func ExtractRecords ¶ added in v0.5.0
func ExtractRecords(streamFunc func(*models.StreamConfig) error)
ExtractRecords begins streaming records from source (sending to ExtractedChan) and start goroutines to extract records
func GenerateSchema ¶
GenerateSchema generates a JSON schema from a record
func RecordMessage ¶ added in v0.6.0
RecordMessage generates a message of the record
func UpdateSchema ¶ added in v0.3.0
UpdateSchema merges the new schema into the existing schema
Types ¶
type TransformationMetrics ¶ added in v0.7.1
type TransformationMetrics struct {
Processed uint64 `json:"processed"`
Skipped uint64 `json:"skipped"`
Filtered uint64 `json:"filtered"` // filtered by bookmark
// contains filtered or unexported fields
}
TransformationMetrics tracks record transformation statistics
Source Files
Click to show internal directories.
Click to hide internal directories.