Hootu Tech Ecosystem and Culture
At Hootu, engineers operate what they build and make the best use of tools, platforms created by the DevOps team to deploy and manage their services be it, CI/CD Pipeline, Orchestration, Deployment Portals, Service Availability or Health Dashboards etc. We also embrace the ideology of “Build for Failure” to ensure that whether it’s streaming VOD or large scale LIVE events, everything runs smoothly, with very little human intervention needed.
However, a world class infrastructure, needs to have a solid security posture as well. The Hootu security team is a few months old and has brought into sharp focus numerous items that make us even stronger than we were before!
Attack Surface and Challenges
To start with, we realized that we were dealing with a fairly broad attack surface:
20+ cloud accounts
Double-digit Kubernetes Clusters
A key focus for us was to ensure that in the preparation for the big upcoming cricket season, we ensured that the team focused on ensuring a high security posture.
From a security point of view, the following were some of the key areas which we doubled down on.
Internal Resources that are available for Public/Anonymous Access: Developers can deploy, operate, manage their services via AWS Console / Kubernetes deployment manifests / Terraform Scripts and we wante dto put in controls on ingress rules for services and resource-based policies for AWS Resources.
DNS Misconfigurations: Developers can create DNS Records which might go into dis-use, however, without a clean-up, it could lead to subdomain takeover issues.
Violation of Principle of Least Privilege: Reviewing various systems that were used to provision access across various tools to developers to ensure that access was retained as needed and only at levels that were required to do the tasks at hand.
Secret Management: Ensure that secrets were handled sensitively
Accountability & Auditing: All assets should have their owners tagged, access policy defined with auditing capability.
Our philosophy has been to create the least friction possible in the current system. It’s easy to stick to a checklist and create process and controls around it but does it solve the security problems? Not in an engineering driven organization. It’s necessary to strike a balance between security and business needs.