Security concerns and DevOps process issues can stall CI/CD operations. DevOps teams are familiar with the frustrations plaguing their pipelines, whether regulatory or process-based. Operational hurdles are all too common in DevOps pipelines, leading to miscommunication between team members and the broader organization. One of the leading operational issues DevOps teams encounter is permission issues. Permission issues are a seemingly small yet significant roadblock to smooth CI/CD pipelines. Fail to address them and the result is a lack of cohesion between development and organizational objectives.
Here’s how to streamline these processes, boost security integration within the broader CI/CD framework, and maintain robust security postures.
Review Pipeline Tools for Permissions and Access
The DevOps life cycle often contains multiple tools, all with different access needs and permissions. As such, this situation is a security nightmare. Jeremy Hess, head of DevRel at secrets management platform Akeyless, called this a “secrets sprawl.”
“The combination of proliferation and decentralization of secrets creates an operational burden, if not a nightmare,” he said. “For organizations that operate in both a cloud-native environment and classic IT infrastructure, a duplication issue is created due to having their own secrets managed with different tools and cloud-native solutions.”
There is also the risk of these tools exposing user credentials and permissions to malicious actors. For instance, configuration tools like Jenkins use plugins to determine access and artifact deployment. Because these tools communicate with other pipeline tools, credential details can often be present in configuration details.
Developer passwords are not visible on the frontend but could accessible from the system backend. Any user with ‘configure’ permissions could request any credential and then inject them into agents. The result is that AWS keys, Git credentials and/or passwords are at risk.
To mitigate this issue, the first step involves deleting hard-coded secrets from CI/CD tool files. Distributing secrets between multiple tool configuration files also reduces the possibility of attack while easing developer and engineer access. Password managers are also a good choice, but validate them for security before implementing a solution.
Practice Least-Privilege Access
Access and permission issues often create a lot of frustration among DevOps teams as they are forced to assign blanket access to the majority of users, regardless of members’ roles or job functions. While this situation encourages rapid development, it also creates massive security issues.
Balancing security with CI/CD needs is tough to get right. That’s where the principle of least privilege comes in. Team members receive access to secrets on a need-to-know basis. Note that this principle applies to everything from apps to systems and connected devices.
While most teams put this principle into practice, they make the mistake of leaving their process intact. The lack of access audits creates DevOps frustration, not the level of access. CISOs should regularly involve DevOps teams when reviewing access to mitigate issues quickly.
For instance, embedding a security role within every delivery team will mitigate access-related risks quickly. The security team member will have insights into risk-based access needs and can quickly approve or reject requests.
Creating an access management repository will also remove any confusion related to role-based access. In addition, record time-based and task-based access permissions in the repository. The result is every DevOps team member understands their access paths before projects get started. It allows them time to offer feedback and request one-off access to sensitive secrets.
Review segmentation rules within your systems when assigning role-based access. Often, these rules will have to change depending on delivery timelines. Involving all stakeholders in these discussions is good practice and prevents frustration down the road.
Implementing OTPs and other authentication factors is also a good idea when validating user access to secrets.
Review OSS Projects
Open source software (OSS) projects are essential to industry growth but might pose security risks if access is mismanaged. Zan Markan, developer advocate at CI platform CircleCI, summarizes the problem aptly.
“Often, the company that initiated and that owns a popular OSS project continues to employ the core contributors,” he said. “They will probably be joined by other regular contributors and maintainers that are not part of that company. And then there is everyone else—anyone who occasionally might contribute a fix or a feature.”
As user access grows, security concerns grow exponentially. Enforcing rigid user-based access is unrealistic and detrimental to an OSS project. However, CISOs or other security-focused managers must review whether sensitive secrets are being passed during builds for pull requests. Monitoring permissions—and who can place requests and roles that review them—will ensure a good level of security.
Establishing machine identity is also critical, given the degree of non-human access pipelines face. Authentication can be based on verifying whether client runtime container attributes match the characteristics of the valid container. Once authenticated, role-based access can take over, limiting access to secrets.
It’s also a good policy to destroy containers and VMs after they’ve been used.
Streamlining DevOps Operations is a Top Security Priority
DevOps is critical to every organization’s success. Access and permission-related issues are common occurrences, but are easily avoided. Reviewing access and establishing a balance between delivery and operational needs is essential to maintaining a competitive edge.
