Wilde-IT
Design and Development of a Multi-Stage, Workflow based Data Processing Pipeline
Focus areas
- Built a Kubernetes-native file integration platform in Go, distributed across core services and remote agents, processing around 10,000 jobs per minute
- Workflow engine based on a state machine with retry, cancellation and flow control between core and agents
- Pluggable protocol adapters for SFTP, SMTP and local filesystems — new sources or targets plug in without touching the core
- gRPC middleware chain for filtering, selective compression, PDF conversion, tokenization and templating
- PostgreSQL-backed job queue with stream-based S3 file transfer for reliable delivery under failure
- Kubernetes CRDs and controllers for workflow management, RBAC with multi-tenancy and enterprise auth via LDAP, OIDC and SAML
Let us talk about your infrastructure
One email is enough. We reply within one business day.