Introduction
Hello. I am @hi120ki, an AI Security engineer at Mercari.
At Mercari, we have rolled out Devin, an AI Agent service, to multiple teams across the company. Devin is a service that can autonomously investigate code, write code, and submit pull requests. However, operating it at an organizational level comes with several management challenges.
In this article, I will introduce how the AI Security team worked together with the AI Agent Platform team to scale the operation of Devin across the entire organization, by building a custom Terraform provider and a set of automated management tools, all powered by the Devin Enterprise API. Through these tools, we established mechanisms for member and permission management, secret rotation, API key lifecycle management, and auditing. We hope that this serves as a blueprint for securing deploying and operating Devin across an enterprise.
Challenges of Enterprise Operations
At Mercari, we use Devin’s Enterprise plan. To operate an AI Agent running in a remote environment at an organizational scale, SSO through Okta, audit logs, permission management, and environment isolation per team were essential requirements, which led us to choose this plan.
In Devin Enterprise, rather than sharing a single Organization as with the Core or Team plans, multiple Organizations are centrally managed through an Enterprise management layer. Mercari has numerous teams spanning multiple business domains, and the information each team handles must be kept isolated and protected. For this reason, we assign Organizations according to team or purpose.
However, in an environment with more than 10 Organizations and a large number of users, the following challenges arise.
Challenges in Permission Management
- Assigning members to Organizations relies on manual operations
- Tracking the state of "who belongs to which Organization" is difficult
Challenges in Secret Management
- Authentication credentials for each third party service must be configured individually per Organization
- Rotating secrets manually across all Organizations is time consuming
Challenges in Access Control
- Devin does not offer expiration management for API keys as a standard feature, creating a risk of long lived, unrotated API keys remaining in each Organization
As Devin adoption grows, the number of Organizations to manage increases, and the burden of these challenges grows with it. Previously, we relied on manual operations through the Web UI, but since late 2025, Devin has released Enterprise API v3, making it possible to automate most management operations through the API. In response, we have built an in-house management platform using Go and GitHub Actions.
Overview of the Devin API
Devin provides its latest Enterprise management API as v3. It allows management of Members, Roles, Secrets, and Knowledge at both the Enterprise and Organization levels. Using the v3 API, we have built the following automated management capabilities:
- Custom Terraform provider
- Bulk secret rotation
- Google Cloud service account key rotation
- Integration with the security management platform
Only API key management uses the v2 API. The v2 API allows creation, retrieval, and deletion of API keys across multiple Organizations, and we use it for the following:
- Periodic invalidation of API keys issued by users
- API key management for internal Agent access to Devin Wiki
These API specifications are documented as REST-format APIs in Devin’s official documentation, complete with detailed specifications for requests and responses, and each function can be invoked by implementing a standard REST API client. For this implementation, we built these REST API clients using Go, which is widely used within Mercari, organizing them so that each API corresponds to a function, making them easy to reuse.
The following sections describe each of these capabilities in detail.
1. Custom Terraform Provider
The core of our management platform is Organization and member management through a custom Terraform provider built with the Terraform Plugin Framework.
At Mercari, Terraform is the standard tool for Infrastructure as Code (IaC) resource management, including Google Cloud, and engineers work with it on a daily basis, which is why we chose this approach. Managing Devin through IaC allows us to insert PR reviews into member additions and permission changes, and makes the state of Organizations and members visible in code. Since no official Terraform provider is available at this time, we built our own.
Users and administrators define each team’s Organization in Terraform. ACU (Agent Compute Unit) limits are also set here to control usage per team. max_cycle_acu_limit sets the overall ACU cap for the Organization, and max_session_acu_limit sets the cap per session, preventing unexpected cost overruns.
resource "devin_organization" "mercari_example_team" {
name = "mercari-example-team"
max_cycle_acu_limit = 500
max_session_acu_limit = 250
}Member assignments to Organizations are also managed declaratively through Terraform.
# Member definition (referenced by email address)
data "devin_member" "mercari_example_team" {
for_each = toset([
"user-1@example.com",
"user-2@example.com",
"user-3@example.com",
])
email = each.value
}
# Assignment to Organization
resource "devin_organization_member" "mercari_example_team" {
for_each = data.devin_member.mercari_example_team
user_id = each.value.user_id
org_id = devin_organization.mercari_example_team.org_id
org_role_id = "mercari_org_member"
}Adding Organizations, changing ACU limits, and adding or removing members all follow the standard development flow of modifying Terraform code, reviewing a PR, and merging. The output of terraform plan clearly shows who will be added to or removed from which Organization, preventing unintended permission changes.
This Terraform provider also manages Devin Knowledge. Knowledge functions similarly to Agent Skills within Devin. In Mercari’s Devin environment, each team is separated into different Organizations and cannot see each other’s usage. While this isolation is desirable from a security standpoint, it makes sharing practical know-how difficult. By making Knowledge manageable through the provider, we enabled the distribution of practical knowhow across teams.
2. Bulk Secret Rotation
Devin launches an independent virtual machine for each Session, so in its initial state, it only has permissions for source code management services such as GitHub. Connecting to cloud environments or ticket management services requires configuring authentication credentials such as API keys individually.
At the same time, as an AI Agent, Devin can freely use any API keys it is given, and members within an Organization can access the file system and shell inside Sessions. This means credentials must be handled with care. At Mercari, we centrally manage the API keys configured in Devin and rotate them at short intervals, ensuring that long lived credentials do not remain on Devin.
However, manual rotation is a heavy burden. Previously, rotating multiple Secrets across numerous Organizations consumed a significant amount of time. When Devin added Secret management to the v3 API in January 2026, it became possible to automate these operations. The current rotation procedure is as follows:
- The Devin administrator rotates credentials in each respective service
- The new credentials are added to a previously created Google Cloud Secret Manager
- The automation is triggered through GitHub Actions
- Rotation is executed, distributing secrets from Secret Manager to each Organization
This allows us to fully automate secret rotation, without any additional effort if new Organizations are created.
3. Google Cloud Service Account Key Rotation
At Mercari, we primarily use Google Cloud, and to retrieve libraries and connect to test environments, we need to grant Google Cloud permissions to Devin. However, Devin currently does not have an OIDC token issuance feature that would allow it to work with Workload Identity Federation, so we must use service account keys.
However, Mercari follows Google Cloud’s official best practices and prohibits the issuance of service account keys across the board through Organization Policy. Therefore, we set up a dedicated Google Cloud Project or Devin excluded from the Organization Policy, with iam.serviceAccountKeyExpiryHours as a compensating control. This ensures that even if automation stops, service account keys are automatically disabled after a fixed period.
On top of this framework, we periodically rotate and assign individual service account keys for each Organization.
4. Integration with the Security Monitoring Platform
One of the requirements for adopting Devin Enterprise was audit logging. At Mercari, Anna from AI Security and Threat Detection and Response team had built integration with our in house security monitoring platform through the Devin v3 API.
We are using an enterprise-level service user with admin access and the Enterprise Audit Logs endpoint which – unlike the v2 endpoint – has pagination. We have a Google Cloud’s Cloud Run Job that pulls and forward all the new logs to a PubSub topic. From there we can analyze the logs and store them in BigQuery for investigation purposes.
5. Periodic Invalidation of API Keys Issued by Users
We enforce API key expiration, by an automation that retrieves all API keys across the entire Enterprise and automatically invalidates any keys that have exceeded a certain period since creation. Devin currently does not provide this as a standard feature.
These API keys are primarily used to connect to Devin MCP. Since source code can be obtained indirectly through API keys, strict management is required. In development environments where multiple AI Agents are in use, situations can arise where credentials remain in configuration files of unused Agents, or where someone sets a personal API key in a custom Agent shared with colleagues and publishes it within the company.
By automatically invalidating API keys after a certain period, we maintain a state where only actively used Agents hold API keys. For Agents shared among multiple people, we have them use API keys managed through Google Cloud Secret Manager as introduced in the next section. This also achieves visibility into the permissions held by each Agent.
6. API Key Management for Internal Agent Access to Devin Wiki
At Mercari, we operate a separate Organization dedicated to Devin Wiki, apart from the development Organizations for each team. Devin Wiki allows retrieval of repository contents and natural language search through Devin MCP.
When an AI Agent directly performs source code exploration, it consumes a large amount of context. By delegating source code investigation to Devin in situations where it is needed, context consumption can be reduced.
However, using Devin MCP requires an API key, and as described in the previous section, keys are automatically invalidated after a certain period. While it is possible to create exception API keys, this cannot completely prevent misuse. Therefore, we built automation that periodically recreates API keys at short intervals and stores them in Google Cloud Secret Manager.
This enables us to centrally manage the service accounts of AI Agents using Devin MCP through Terraform, providing visibility into usage, while also preventing misuse through periodic key recreation.
resource "google_secret_manager_secret" "shared_wiki_api_key" {
secret_id = "shared-wiki-api-key"
}
resource "google_secret_manager_secret_iam_member" "shared_wiki_api_key" {
for_each = toset(local.accessor_service_accounts_shared_wiki_api_key)
secret_id = google_secret_manager_secret.shared_wiki_api_key.secret_id
role = "roles/secretmanager.secretAccessor"
member = "serviceAccount:${each.value}"
}
locals {
accessor_service_accounts_shared_wiki_api_key = [
"agent-1@---.iam.gserviceaccount.com",
"agent-2@---.iam.gserviceaccount.com",
]
}CI Pipeline for Unified Orchestration
All of these management operations are automated through GitHub Actions. When building custom management tools for SaaS administration, long term maintenance is unavoidable. Considering handoffs during organizational changes, it is necessary to keep dependencies small and choose technologies and platforms that are easy to maintain.
While Secret Manager and service accounts reside on Google Cloud, we chose GitHub Actions for execution. Since automation within the repository runs directly without deployment, maintenance effort is reduced. By not holding unnecessary cloud resources, we also keep costs low and reduce the mental burden during management and handoffs. In addition to scheduled runs, we support manual triggers (workflow_dispatch), allowing immediate secret rotation in emergencies.
On the other hand, because GitHub Actions can be freely executed, we strictly configure permission management and branch protection settings. For credential retrieval, we use Google Cloud Workload Identity Federation to securely access service accounts and Secret Manager from GitHub Actions.
Conclusion
In operating Devin Enterprise at scale, we supplemented management requirements that could not be covered by standard features alone with custom tools built using the v2 and v3 APIs. This helped us overcome management challenges that had previously relied on manual work, enabling us to provide and properly manage many Organizations in parallel.
The currently available Devin v3 API already includes the endpoints required for Enterprise administration. Going forward, we plan to continue automating the safe management of a wider range of resources as Devin’s capabilities expand.
We hope this article will be helpful to those facing similar challenges.
If you are interested in AI and LLM adoption and security initiatives at Mercari, please visit Mercari’s careers page.
