Welcome to Knowledge Base!

KB at your finger tips

This is one stop global knowledge base where you can learn about all the products, solutions and support features.

Categories
All

Database-MongoDB

Deploy a Cluster through the API — MongoDB Cloud Manager

Deploy a Cluster through the API¶

On this page

  • Prerequisites
  • Examples
  • Variables for Cluster Creation API Resources
  • Prerequisites
  • Procedures
  • Next Steps

This tutorial manipulates the Cloud Manager Administration API’s automation configuration to deploy a sharded cluster that is owned by another user. The tutorial first creates a new project, then a new user as owner of the project, and then a sharded cluster owned by the new user. You can create a script to automate these procedures for use in routine operations.

To perform these steps, you must have sufficient access to Cloud Manager. A user with the Project Owner role has sufficient access.

The procedures install a cluster with two shards . Each shard comprises a three-member replica set . The tutorial installs one mongos and three config servers . Each component of the cluster resides on its own server, requiring a total of 10 hosts.

The tutorial installs the MongoDB Agent on each host.

Prerequisites¶

Provision ten hosts to serve the components of the sharded cluster . For host requirements, see the Production Notes in the MongoDB manual.

Each host must provide its MongoDB Agent with full networking access to the hostnames and ports of the MongoDB Agents on all the other hosts. Each agent runs the command hostname -f to self-identify its hostname and port and report them to Cloud Manager.

Tip

To ensure agents can reach each other, provision the hosts using Automation . This installs the MongoDB Agents with correct network access. Use this tutorial to reinstall the Automations on those machines.

Examples¶

As you work with the API , you can view examples on the GitHub example page.

Variables for Cluster Creation API Resources¶

The API resources use one or more of these variables. Replace these variables with your desired values before calling these API resources.

Name Type Description
PUBLIC-KEY string Your public API Key for your API credentials.
PRIVATE-KEY string Your private API Key for your API credentials.
cloud.mongodb.com string URL of your Cloud Manager instance.
GROUP-ID string Unique identifier of your project from your Project Settings .

Prerequisites¶

  • Configure API Access to enable you to use the API.
  • Complete the MongoDB Agent Prerequisites .

Procedures¶

Create the Group and the User through the API¶

1

Use the API to create a project.¶

Use the Cloud Manager Administration API to send a projects document to create the new project.

curl --user "{PUBLIC-KEY}:{PRIVATE-KEY}" --digest \
     --header "Content-Type: application/json" \
     --request POST "https://cloud.mongodb.com/api/public/v1.0/groups?pretty=true" \
     --data '
       {
         "name": "{GROUP-NAME}"
       }'

The API returns a document that includes the project’s agentApiKey and id .

2

Record the values of agentApiKey and id in the returned document.¶

Record these values for use in this procedure and in other procedures in this tutorial.

3

Use the API to create a user in the new project.¶

Use the /users endpoint to add a user to the new project.

The body of the request should contain a users JSON document with the user’s information.

Set the user’s roles.roleName to GROUP_OWNER and the user’s roles.groupId set to the new group’s‘ id .

curl --user "{PUBLIC-KEY}:{PRIVATE-KEY}" --digest \
     --header 'Accept: application/json' \
     --header "Content-Type: application/json" \
     --request POST "https://cloud.mongodb.com/api/public/v1.0/users?pretty=true" \
     --data '
       {
          "username": "<new_user@example.com>",
          "emailAddress": "<new_user@example.com>",
          "firstName": "<First>",
          "lastName": "<Last>",
          "password": "<password>",
          "roles": [{
            "groupId": "{PROJECT-ID}",
            "roleName": "GROUP_OWNER"
          }]
       }'

Install the MongoDB Agent on each Provisioned Host¶

1

Complete the MongoDB Agent installation procedure on each host.¶

To learn how to install the MongoDB Agent, follow the procedure for the appropriate platform .

2

Confirm the initial state of the automation configuration.¶

When the MongoDB Agent first runs, it downloads the mms-cluster-config-backup.json file, which describes the desired state of the automation configuration .

On one of the hosts, navigate to /var/lib/mongodb-mms-automation/ and open mms-cluster-config-backup.json . Confirm that the file’s version field is set to 1 . Cloud Manager automatically increments this field as changes occur.

Deploy the New Cluster¶

To add or update a deployment, retrieve the configuration , make changes as needed, and send the updated configuration though the API to Cloud Manager.

The following procedure deploys an updated automation configuration through the API :

1

Retrieve the automation configuration from Cloud Manager.¶

  1. Use the automationConfig resource to retrieve the configuration. Issue the following command, replacing the placeholders with the Variables for Cluster Creation API Resources.

    curl --user "{PUBLIC-KEY}:{PRIVATE-KEY}" --digest \
         --request GET "https://cloud.mongodb.com/api/public/v1.0/groups/{PROJECT-ID}/automationConfig?pretty=true" \
         --output currentAutomationConfig.json
    
  2. Validate the downloaded Automation Configuration file.

    Compare the version field of the currentAutomationConfig.json with that of the Automation Configuration backup file, mms-cluster-config-backup.json . The version value is the last element in both JSON documents. You can find this file on any host running the MongoDB Agent at:

    • Linux and macOS: /var/lib/mongodb-mms-automation/mms-cluster-config-backup.json
    • Windows: %SystemDrive%\MMSAutomation\versions\mms-cluster-config-backup.json

    If the version values match, you are working with the current version of the Automation Configuration file.

2

Create the top level of the new automation configuration.¶

Create a document with the following fields. As you build the configuration document, refer the description of an automation configuration for detailed explanations of the settings. For examples, see the MongoDB Labs page.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
{
    "options": {
        "downloadBase": "/var/lib/mongodb-mms-automation",
    },
    "mongoDbVersions": [],
    "monitoringVersions": [],
    "backupVersions": [],
    "processes": [],
    "replicaSets": [],
    "sharding": []
}
4

Add the Monitoring to the automation configuration.¶

In the monitoringVersions.hostname field, enter the hostname of the server where Cloud Manager should install the Monitoring. Use the fully qualified domain name that running hostname -f on the server returns, as in the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
"monitoringVersions": [
  {
    "hostname": "<server_x.example.com>",
    "logPath": "/var/log/mongodb-mms-automation/monitoring-agent.log",
    "logRotate": {
      "sizeThresholdMB": 1000,
      "timeThresholdHrs": 24
    }
  }
]

This configuration example also includes the logPath field, which specifies the log location, and logRotate , which specifies the log thresholds.

5

Add the servers to the automation configuration.¶

This sharded cluster has 10 MongoDB instances, as described in the Deploy a Cluster through the API , each running on its own server. Thus, the automation configuration’s processes array will have 10 documents, one for each MongoDB instance.

The following example adds the first document to the processes array. Replace <process_name_1> with any name you choose, and replace <server1.example.com> with the FQDN of the host.

Add 9 documents: one for each MongoDB instance in your sharded cluster.

Specify the args2_6 syntax for the processes.<args> field. See MongoDB Settings that Automation Supports for more information.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
"processes": [
  {
    "version": "4.0.6",
    "name": "<process_name_1>",
    "hostname": "<server1.example.com>",
    "logRotate": {
      "sizeThresholdMB": 1000,
      "timeThresholdHrs": 24
    },
    "authSchemaVersion": 5,
    "featureCompatibilityVersion": "4.0",
    "processType": "mongod",
    "args2_6": {
      "net": {
        "port": 27017
      },
      "storage": {
        "dbPath": "/data/"
      },
      "systemLog": {
        "path": "/data/mongodb.log",
        "destination": "file"
      },
      "replication": {
        "replSetName": "rs1"
      }
    }
  },
]
6

Add the sharded cluster topology to the automation configuration.¶

Add two replica set documents to the replicaSets array. Add three members to each document.

Example

This section adds one replica set member to the first replica set document:

Important

You must include "protocolVersion": 1 in the root document for each replica set.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
"replicaSets": [
  {
    "_id": "rs1",
    "members": [
      {
        "_id": 0,
        "host": "<process_name_1>",
        "priority": 1,
        "votes": 1,
        "slaveDelay": 0,
        "hidden": false,
        "arbiterOnly": false
      }
    ],
    "protocolVersion": 1
  }
]

In the sharding array, add the replica sets to the shards, and add the config server replica set name, as in the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
"sharding": [
  {
    "shards": [
      {
        "tags": [],
        "_id": "shard1",
        "rs": "rs1"
      },
      {
        "tags": [],
        "_id": "shard2",
        "rs": "rs2"
      }
    ],
    "name": "sharded_cluster_via_api",
    "configServerReplica": "rs-config",
    "collections": []
  }
]
7

Send the updated automation configuration.¶

Use the automationConfig resource to send the updated automation configuration.

Issue the following command with path to the updated configuration document and replace the placeholders with the Variables for Cluster Creation API Resources.

curl --user "{PUBLIC-KEY}:{PRIVATE-KEY}" --digest \
     --header "Content-Type: application/json"
     --request PUT "https://cloud.mongodb.com/api/public/v1.0/groups/{PROJECT-ID}/automationConfig?pretty=true" \
     --data @currentAutomationConfig.json

Upon successful update of the configuration, the API returns the HTTP 200 OK status code to indicate the request has succeeded.

8

Confirm successful update of the automation configuration.¶

Retrieve the automation configuration from Cloud Manager and confirm it contains the changes. To retrieve the configuration, issue the following command, replacing the placeholders with the Variables for Cluster Creation API Resources.

curl --user "{PUBLIC-KEY}:{PRIVATE-KEY}" --digest \
     --request GET "https://cloud.mongodb.com/api/public/v1.0/groups/{PROJECT-ID}/automationConfig?pretty=true"
9

Verify that the configuration update is deployed.¶

Use the automationStatus resource to verify the configuration update is fully deployed. Issue the following command, replacing the value placeholders given in Variables for Cluster Creation API Resources:

curl --user "{PUBLIC-KEY}:{PRIVATE-KEY}" --digest \
     --request GET "https://cloud.mongodb.com/api/public/v1.0/groups/{PROJECT-ID}/automationStatus?pretty=true"

The curl command returns a JSON object containing the processes array and the goalVersion key and value. The processes array contains a document for each server that hosts a MongoDB instance. The new configuration is successfully deployed when all lastGoalVersionAchieved fields in the processes array equal the value specified for goalVersion .

Example

In this response, processes[2].lastGoalVersionAchieved is behind goalVersion . This indicates that the MongoDB instance at server3.example.com is running one version behind the goalVersion . Wait several seconds and issue the curl command again.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
{
  "goalVersion": 2,
  "processes": [{
    "hostname": "server1.example.com",
    "lastGoalVersionAchieved": 2,
    "name": "ReplSet_0",
    "plan": []
  }, {
    "hostname": "server2.example.com",
    "lastGoalVersionAchieved": 2,
    "name": "ReplSet_1",
    "plan": []
  }, {
     "hostname": "server3.example.com",
     "lastGoalVersionAchieved": 1,
     "name": "ReplSet_2",
     "plan":[]
  }]
}

To view the new configuration in the Cloud Manager console, click Deployment .

Next Steps¶

To make an additional version of MongoDB available in the cluster, see Update the MongoDB Version of a Deployment .

Fix Replication Lag — MongoDB Cloud Manager

Fix Replication Lag¶

On this page

  • Alert Conditions
  • Common Triggers
  • Fix the Immediate Problem
  • Implement a Long-Term Solution
  • Monitor Your Progress

At time T , the last write operation applied on the specified secondary of replica set ABC was behind the most recent operation applied on the primary .

Alert Conditions¶

You can configure alert conditions in the project-level alert settings page to trigger alerts.

To learn more about the alert condition, see Replication Lag is .

Common Triggers¶

  • An idle replica set. The reported replication lag is actually just the time since the last write. Replication lag is calculated between the last operation time on the primary and the time of the last operation received by the secondary. If a replica set is only written to once every 10 minutes, the replication lag will be 10 minutes just after the write is made to the primary and just prior to the next write being replicated to the secondary.
  • The secondary is under-provisioned, which means it needs more allocated resources, and cannot keep up with the primary (common if using secondaries for read scaling).
  • There is insufficient bandwidth, or some other networking problem, between the primary and secondary.

Fix the Immediate Problem¶

  • Adjust the settings for this alert to only trigger if the replication lag persists for longer than 2 minutes. This will reduce the chances of a false positive.
  • Resolve networking issues between the primary and secondary.

To learn more, see Troubleshoot Replica Sets in the MongoDB manual.

Implement a Long-Term Solution¶

  • Increase bandwidth between the primary and secondary.
  • Move (or upgrade in place) the secondary to a machine that is identically (or better) provisioned to the current primary.

Monitor Your Progress¶

View the following charts to monitor your progress:

  • Network

    Monitor network metrics to track network performance.

  • Replication Headroom

    Monitor replication headroom to determine whether the secondary might fall off the oplog.

  • Replication Lag

    Monitor replication lag to determine whether the secondary might fall off the oplog.

To learn more, see View Deployment Metrics .

Read article

Configure MongoDB Agent for LDAP — MongoDB Cloud Manager

Configure MongoDB Agent for LDAP¶

On this page

  • Prerequisites
  • Create and Configure User in MongoDB

If your MongoDB deployment enforces access control, the MongoDB Agent must authenticate to MongoDB as a user with the proper access. If you use Automation , Cloud Manager takes care of this for you.

MongoDB Enterprise supports simple and SASL binding to LDAP servers via saslauthd and operating system libraries:

  • MongoDB Enterprise for Linux can bind to an LDAP server either via saslauthd or via operating system libraries.
  • MongoDB Enterprise for Windows can bind to an LDAP server via the operating system libraries.

MongoDB Agent support authenticating to MongoDB instances using LDAP .

Note

With Automation, Cloud Manager manages MongoDB Agent authentication for you. To learn more about authentication, see Enable LDAP Authentication for your Cloud Manager Project .

Prerequisites¶

Configure Deployments to Use Authentication¶

The MongoDB Agent interacts with the MongoDB databases in your deployment as a MongoDB user would. As a result, you must configure your MongoDB deployment and the MongoDB Agent to support authentication.

You can specify the deployment’s authentication mechanisms when adding the deployment, or you can edit the settings for an existing deployment. At minimum, the deployment must enable the authentication mechanism you want the MongoDB Agent to use. The MongoDB Agent can use any supported authentication mechanism .

Set Require TLS Certificate Environment Variable¶

On the MongoDB Agent hosts, you must set the TLS_REQCERT environment variable to demand .

Example

In a Red Hat Enterprise Linux host, open the /etc/openldap/ldap.conf file and add the following setting and value:

TLS_REQCERT demand

You can use your application to set this environment variable.

Considerations¶

If Automation does not manage your deployment, you must configure LDAP authentication separately for each function.

To configure LDAP authentication , add a host or edit an existing host’s configuration.

Create and Configure User in MongoDB¶

To automate MongoDB instances that use LDAP authentication, add a MongoDB user that possesses the required roles and privileges to the $external database in MongoDB. The $external database allows mongod to consult an external source, such as an LDAP server, to authenticate.

Use the following commands to create the users from mongosh :

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
   db.getSiblingDB("$external").createUser(
     {
       user : "<username>",
       roles : [ 
         { role : "clusterAdmin", db : "admin" },
         { role : "readWriteAnyDatabase", db : "admin" },
         { role : "userAdminAnyDatabase", db : "admin" },
         { role : "dbAdminAnyDatabase", db : "admin" },
         { role : "backup", db : "admin" },
         { role : "restore", db : "admin" }
       ]
     }
   )

To learn more about the required access, see Required Access for MongoDB Agent .

Read article

MongoDB Cloud Manager — MongoDB Cloud Manager

MongoDB Cloud Manager ¶

What’s New

  • Support for write operation through the Interact with Your Data .

Welcome to the documentation for MongoDB Cloud Manager. Engineered by the team who develops MongoDB, Cloud Manager provides a complete package for managing MongoDB deployments.

Cloud Manager Overview
Describes Cloud Manager.
Create Deployments
Provision servers and create MongoDB deployments.
Manage Deployments
Manage and update your MongoDB deployments.
Migrate Deployments
Migrate your MongoDB deployments to MongoDB Atlas.
Monitor Your Deployments
Monitor your MongoDB deployments and manage alerts.
Backup and Restore
Initiate and restore backups.
Security
Describes Cloud Manager security features.
MongoDB Agent
View and manage Cloud Manager MongoDB Agent.
Organizations and Projects
Manage Cloud Manager organizations, projects, and users.
Account Management
Manage your Cloud Manager user account.
API
Manage Cloud Manager through the API.
Troubleshooting
Troubleshooting advice for common issues.
Frequently Asked Questions
Common questions about the operation and use of Cloud Manager.
Reference
Reference material for Cloud Manager components and operations.
Release Notes
Changelogs and notes on Cloud Manager releases.
Cloud Manager Licensing
Cloud Manager special licensing.
Read article

Remove a Process from Monitoring — MongoDB Cloud Manager

Remove a Process from Monitoring¶

On this page

  • Monitored Processes
  • Remove a Process from Monitoring

Monitored Processes¶

Removing a process from monitoring means Cloud Manager no longer displays its status or tracks its metrics. You must terminate the deployment’s backups before you can remove a monitored deployment.

Remove a Process from Monitoring¶

Follow this procedure to remove one monitored process from Cloud Manager.

1
2

Click Modify in the cluster from which you want to remove a process.¶

3

In Member Configuration , click the ellipsis icon next to the process that you want to remove and select Remove from Cluster

For replica sets, select Remove from Replica Set . For mongod processes in a sharded replica set, select Remove From Shard . For mongos processes, select Remove from Cluster .

4

Click Remove from Cluster in the verification dialog.¶

For replica sets, select Remove from Replica Set . For mongod processes in a sharded replica set, select Remove From Shard . For mongos processes, select Remove from Cluster .

5

Click Save

6

Click Review & Deploy to review your changes.¶

7

Click Confirm and Deploy to deploy your changes.¶

The process that you removed earlier now appears as a standalone process. However, mongos processes are automatically removed from the cluster and do not appear.

8

Click the ellipsis next to the standalone process, and select Remove from Cloud Manager.¶

Read article

Install the MongoDB Agent to Manage Deployments — MongoDB Cloud Manager

Install the MongoDB Agent to Manage Deployments¶

  • Windows
  • Ubuntu/Debian
  • RHEL/CentOS/SLES/AMZ
  • Linux

You have a project and want to install the MongoDB Agent to manage your MongoDB deployments. You can also monitor and back up your MongoDB deployments following this workflow.

Review the Prerequisites First

Please review the MongoDB Agent Prerequisites before installing the MongoDB Agent.

Use this procedure to install the MongoDB Agent on x86_64 architecture running Microsoft Windows:

1

Log in to the Cloud Manager.¶

2

Begin or Manage a Deployment.¶

Your project has no deployments and you see a prompt to get started.

  1. From the Build New Deployment menu, select the type of deployment you wish to add:

    • New Sharded Cluster
    • New Replica Set
    • New Standalone
  2. If, on the Create New <Deployment> page, Cloud Manager cannot detect any hosts with the MongoDB Agent installed, you see a banner displayed at the top of the Create New <Deployment> page.

    Click the see instructions link.

  3. From the Select Your Server’s Operating System menu, click Windows - MSI .

  4. Click Next .

3

Download the MongoDB Agent.¶

The MongoDB Agent Installation Instructions box displays the following information:

  • Project ID (Required for binding to a project)

  • API Key

    If you do not have an API Key, click plus icon Generate Key .

Some or all of these values are required in a later step. Copy these values then store them where you can access them later.

4

Install the MongoDB Agent.¶

  1. After the MSI downloads, double-click:

    mongodb-mms-automation-agent-<VERSION>.windows_x86_64.msi

  2. If a security warning appears, click Run .

  3. At the Configuration/Log Folder step

    Provide the directory into which these files are saved.

  4. At the Key Type step, select Agent API Key to bind to a specific project .

  5. Enter the appropriate agent keys.

    Note

    These keys are provided in the MongoDB Agent Installation Instructions modal described in the previous step.

    • Type your Project ID into the Project ID field.
    • Type your Agent API Key into the Agent API Key field.
  6. At the MongoDB Paths step, specify the Log and Backup directories

  7. At the Windows Firewall Configuration step, click your preferred firewall configuration.

    If you click Configure firewall rules allowing access from only the specified |ipaddr| addresses. , type the necessary IPv4 addresses into the provided box.

  8. (Conditional) Windows enables Stealth Mode for the Windows Firewall by default. If you have not disabled it on the MongoDB host on which you are installing the MongoDB Agent, you are asked to disable it. Stealth Mode significantly degrades the performance and capability of the MongoDB Agent. Click Disable Stealth Mode .

  9. (Conditional) Windows does not enable Disk Performance Counters by default. If you have not enabled Disk Performance Counters for the MongoDB host, click Enable Disk Performance Counters . The MongoDB Agent uses these counters for some of its hardware monitoring activities.

  10. Click Install .

  11. Click Finish once setup is complete.

5

Finish the installation of the MongoDB Agent.¶

In the Install Agent Instructions modal:

  1. Click Verify Agent to make sure the MongoDB Agent can connect to Cloud Manager.
  2. Click Done .
6

(Optional) Change the location of the MongoDB Agent binaries and configuration backup files.¶

By default, the MongoDB binaries and Cloud Manager configuration backup file are located in %SystemDrive%\MMSMongoDB\versions .

Note

Windows sets the %SystemDrive% environment variable to the drive on which you installed Windows. By default, you would be directed to install Windows on the C: drive. To find your %SystemDrive% , issue with following command from PowerShell:

get-childitem env:SystemDrive

If you want to store these files in a different directory, follow these procedures:

To change the location of the MongoDB Agent Binaries

  1. Click Deployment , then Agents , and then Downloads & Settings .

  2. Below the Download Directory heading, click edit icon to the right of the path shown in Download Directory (Windows) .

  3. Change the path to the new path you want.

  4. Click Save .

  5. Create the new directory you specified on each host that runs an MongoDB Agent. Use Windows Explorer to move the file or issue the following command from a Command Prompt or PowerShell:

    md \<newPath>
    

    Important

    Make sure the system user that runs the MongoDB Agent can write to this new directory. This is usually the SYSTEM user, which requires no additional configuration unless you changed the user.

To change the location of the MongoDB Agent configuration backup

  1. Open the MongoDB Agent configuration file in your preferred text editor.

  2. Change the mmsConfigBackup setting to the new path for the configuration backup file. Replace X with the drive letter on which your backup is stored.

    mmsConfigBackup=X:\<newPath>\mms-cluster-config-backup.json
    
  3. Save the MongoDB Agent configuration file.

  4. Move the configuration backup file to the new directory. Use Windows Explorer to move the file or issue the following command from a Command Prompt or PowerShell:

    move %SystemDrive%\MMSMongoDB\versions\mms-cluster-config-backup.json \<newPath>
    
7

Repeat the installation procedure for each MongoDB host.¶

Read article