Data silos are the enemy of enterprise efficiency. When we set out to build Omnia, we realized the biggest hurdle wasn't building the UI—it was unifying the underlying data models of tools like Salesforce, Jira, Workday, and Zendesk.
This post explores how we engineered a "Universal Schema" that bridges these gaps, allowing teams to query and mutate data across platforms as if it all lived in a single database.
The Problem with Traditional Integrations
Most integration platforms rely on point-to-point webhook mapping. If you want to sync a Salesforce Lead to a Zendesk Ticket, you map the fields manually. If you add Jira into the mix, you map them again. The complexity grows exponentially: O(n^2).
"Integration logic should not be coupled to the endpoints it connects."
Our Solution: The Middle Layer
Instead of mapping tools to each other, we mapped all tools to a centralized schema. We designed a GraphQL-like interface that defines universal entities:
ContactOrganizationTaskProjectInvoice
The Architecture
We built the core ingestion engine in Go, utilizing heavily concurrent workers.
// Simplified worker pool example
func processEvents(jobs <-chan Event, results chan<- Result) {
for event := range jobs {
normalizedData := normalizeToSchema(event)
results <- store(normalizedData)
}
}
- Ingestion Layer: Webhooks are caught, parsed, and pushed to a Kafka topic.
- Normalization Engine: Rust-based microservices pick up the events and map vendor-specific JSON to our internal Protobuf schemas.
- Graph Store: The normalized entities are stored in Neo4j, allowing us to maintain the relational context across completely unrelated applications.
What's Next?
We are currently open-sourcing the base normalization maps. By standardizing how software communicates at the entity level, we can eliminate the brittle point-to-point integrations that plague modern IT teams.