diff --git a/.gitbook/assets/cli-download.png b/.gitbook/assets/cli-download.png new file mode 100644 index 00000000..a33e4fde Binary files /dev/null and b/.gitbook/assets/cli-download.png differ diff --git a/.gitbook/assets/cli-download2.png b/.gitbook/assets/cli-download2.png new file mode 100644 index 00000000..ba65a701 Binary files /dev/null and b/.gitbook/assets/cli-download2.png differ diff --git a/SUMMARY.md b/SUMMARY.md index a76e5195..90658bd6 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -29,6 +29,10 @@ * [Importing external recipes](how-to/importing-external-recipes.md) * [Terraform Cloud integration](how-to/terraform-cloud-integration.md) +## Moderne CLI + +* [Getting started with the Moderne CLI](/cli/cli-intro.md) + ## Releases * [Versions of plugins and agent](releases/releases.md) diff --git a/cli/cli-intro.md b/cli/cli-intro.md new file mode 100644 index 00000000..db50c77e --- /dev/null +++ b/cli/cli-intro.md @@ -0,0 +1,255 @@ +# Getting started with the Moderne CLI + +The Moderne CLI is a command line tool that simplifies the process of building and publishing [Lossless Semantic Tree](https://docs.moderne.io/concepts/lossless-semantic-trees) (LST) artifacts to your artifact repository. Instead of having to manually set up each repository individually, you can use the CLI to configure thousands of repositories at once. + +To help you get started with the Moderne CLI, this tutorial will: + +* [Provide you with detailed installation instructions](#installation) +* [Walk you through each of the commands and their parameters](#commands) + +## Installation + +There are two ways to obtain the Moderne CLI: via the SaaS (**recommended**) or via a `curl` command. + +{% tabs %} +{% tab title="SaaS" %} +1. Go to the [Moderne SaaS](https://public.moderne.io/) and sign in. +2. Click on the `?` in the top right corner and then select `Moderne CLI` from the `Tools` menu: + + ![](.gitbook/assets/cli-download.png) +3. You can then either press the `Download Latest` button or you can install it directly through the command line by copying the `curl` command at the bottom of the modal: + + ![](.gitbook/assets/cli-download2.png) +{% endtab %} + +{% tab title="curl command" %} +From your command line, please run the command that corresponds to the OS you're using. + +_If you want to download a different version of the CLI, replace the **two** versions (`v#.#.#`) in the curl request with the_ [_CLI version_](https://github.com/moderneinc/moderne-cli/releases) _you want._ + +**Mac OS** + +```shell +curl --request GET https://pkgs.dev.azure.com/moderneinc/moderne_public/_packaging/moderne/maven/v1/io/moderne/moderne-cli-macos-tar/v0.0.55/moderne-cli-macos-tar-v0.0.55 > mod.tar.gz + +tar -zxvf mod.tar.gz +``` + +**Linux** + +```shell +curl --request GET https://pkgs.dev.azure.com/moderneinc/moderne_public/_packaging/moderne/maven/v1/io/moderne/moderne-cli-linux-tar/v0.0.55/moderne-cli-linux-tar-v0.0.55 > mod.tar.gz + +tar -zxvf mod.tar.gz +``` + +**Windows** + +```shell +curl --request GET https://pkgs.dev.azure.com/moderneinc/moderne_public/_packaging/moderne/maven/v1/io/moderne/moderne-cli-windows/v0.0.55/moderne-cli-windows-v0.0.55 > mod +``` + +**JDK (8+)** + +```shell +curl --request GET https://pkgs.dev.azure.com/moderneinc/moderne_public/_packaging/moderne/maven/v1/io/moderne/moderne-cli-zip/v0.0.55/moderne-cli-zip-v0.0.55 > moderne-cli.zip + +unzip moderne-cli.zip +``` +{% endtab %} +{% endtabs %} + +### Add an alias to run the CLI more easily + +Once you have downloaded and extracted the CLI to the location you want it, it's a good idea to add an [alias](https://en.wikipedia.org/wiki/Alias_(command)) to the Moderne CLI so that you can type `mod ` instead of having to type `/path/to/moderne-cli `. If you use bash or zsh for your shell, you can modify your `.bashrc` or `.zshrc` file and add something similar to: + +```shell +alias mod='/path/to/moderne-cli' +``` + +You can then open a new terminal or type `source ~/.zshrc` to begin using it right away. + +## Commands + +* [Build](#build) +* [Publish](#publish) +* [Connect Jenkins](#connect-jenkins) +* [Connect GitHub](#connect-github) + +### Build + +The `build` command allows you to manually create the LST artifacts with Group Artifact Version coordinates for a specific project. Once created, you can manually upload the artifacts to your artifact library so that Moderne can ingest them. + +This command is typically used for _debugging purposes_. In general, you should favor setting up a [Jenkins](#connect-jenkins) or [Github](#connect-github) connection for building, publishing, and ingesting LST artifacts in bulk. + +If the command executes successfully, the LST artifact for each project will be stored in one of three places: + +* For non-Java projects, a `.moderne/` directory will be created and used +* For Gradle projects, the `build` directory will be used +* For Maven projects, the `target` directory will be used + +**Example:** + +```sh +mod build --path=/path/to/project +``` + +#### Parameters + +| **Parameter Name** | **Description** | +| ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `--path` |

Required

 

The path on disk to the project directory you want to be ingested into the Moderne platform. Typically a checked-out copy of a Git repository.

| +| `--additionalBuildArgs` |

Optional

 

Additional arguments that are added to the Maven or Gradle build command.

| +| `--desiredStyle` |

Optional

 

The OpenRewrite style that should be applied to the code whenever a recipe is run. If a style isn't specified, OpenRewrite will detect the existing style and use that.

| +| `--gradlePluginVersion` |

Optional

 

The version of the Moderne Gradle plugin that should be used to build the artifacts.

| +| `--mirrorUrl` |

Optional

 

For Gradle projects, this can be specified as a Maven repository cache/mirror to check for artifacts before any other repositories are looked at.

| +| `--mvnPluginVersion` |

Optional

 

The version of the Moderne Maven plugin that should be used to build the artifacts.

| +| `--mvnSettingsXml` |

Optional

 

The path to the settings.xml file that should be read and used for Maven builds.

| +| `--recursive` |

Optional

 

Specifies whether or not projects should be looked for recursively. If this parameter is included, it will start at the --path directory and recursively look for all git projects to build.

| + +### Publish + +The publish command allows you to manually build and publish LST artifacts for a specific project. Once published to your artifact repository, Moderne will be able to ingest them and they will, in turn, be usable inside of the SaaS. + +This command is typically used for _debugging purposes_. In general, you should favor setting up a [Jenkins](#connect-jenkins) or [Github](connect-github) connection for building, publishing, and ingesting LST artifacts in bulk. + +This command will begin by executing the [build](#build-command) command and, if that's successful, it will then attempt to upload the artifacts to the artifact repository you specified. + +**Example:** + +```sh +mod publish --path=/path/to/project \ + --publishUser=some-username + --publishPwd=myPassword \ + --publishUrl=https://some-artifact-repo.com \ +``` + +#### Parameters + +| **Parameter Name** | **Description** | +| ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `--path` |

Required

 

The path on disk to the project directory you want to be ingested into the Moderne platform. Typically a checked-out copy of a Git repository.

| +| `--publishUser` |

Required

 

The user that has permission to upload LST artifacts.

| +| `--publishPwd` |

Required

 

The password for the user that has permission to upload LST artifacts.

| +| `--publishUrl` |

Required

 

The URL of the Maven formatted repository where LST artifacts should be uploaded to.

| +| `--additionalBuildArgs` |

Optional

 

Additional arguments that are added to the Maven or Gradle build command.

| +| `--desiredStyle` |

Optional

 

The OpenRewrite style that should be applied to the code whenever a recipe is run. If a style isn't specified, OpenRewrite will detect the existing style and use that.

| +| `--gradlePluginVersion` |

Optional

 

The version of the Moderne Gradle plugin that should be used to build the artifacts.

| +| `--mirrorUrl` |

Optional

 

For Gradle projects, this can be specified as a Maven repository cache/mirror to check for artifacts before any other repositories are looked at.

| +| `--mvnPluginVersion` |

Optional

 

The version of the Moderne Maven plugin that should be used to build the artifacts.

| +| `--mvnSettingsXml` |

Optional

 

The path to the settings.xml file that should be read and used for Maven builds.

| +| `--recursive` |

Optional

 

Specifies whether or not projects should be looked for recursively. If this parameter is included, it will start at the --path directory and recursively look for all git projects to build.

| +| `--skipBuild` |

Optional

 

If this parameter is included, the build command will not be run before this publish command.

| +| `--skipSSL` |

Optional

 

If this parameter is included, SSL verification will be skipped.

| + +### Connect Jenkins + +The `connect jenkins` job allows you to create Jenkins Jobs in bulk. For each configured repository, a Jenkins Job will be made that builds and publishes LST artifacts to your artifact repository on a regular basis. + +{% hint style="info" %} +If you are a CloudBees CI authenticated user, you will also need these permissions: + +* Overall/System Read access. This is needed to get the list of plugins and their versions. + * `GET /pluginManager/api/json` +* Create, Configure, Read folders and Jobs. + * `POST /createItem` + * `GET /job/$folder/api/json` + * `GET /job/$folder/job/$item/api/json` +* (Optionally) Delete jobs. This is only required if `--deleteSkipped` is selected. + * `POST /job/$folder/job/$item/doDelete` + +For more details around these permissions, please see the [CloudBees docs](https://docs.cloudbees.com/docs/cloudbees-ci/latest/cloud-secure-guide/delegating-administration-modern#\_overallsystem\_read). +{% endhint %} + +**Example:** + +```sh +mod connect jenkins --controllerUrl=https://jenkins.company-name.ninja \ + --fromCsv=/path/to/repos.csv \ + --gitCredsId=username-pat \ + --jenkinsUser=some-username + --publishCredsId=artifactory \ + --publishUrl=https://artifact-place.com/artifactory/moderne-ingest \ +``` + +#### Parameters + +| **Parameter Name** | **Description** | +| ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `--controllerUrl` |

Required

 

The URL of the Jenkins controller that will create the jobs. Typically this is the URL of your Jenkins instance.

| +| `--fromCsv` |

Required

 

The location of the CSV file containing the list of repositories that should be ingested. One Jenkins Job will be made for each repository.

Follows the schema of: [repoName,branch,javaVersion,style,buildAction,skip,skipReason].

| +| `--gitCredsId` |

Required

 

The ID of the Jenkins credentials needed to clone the provided list of repositories.

| +| `--jenkinsUser` |

Required

 

The Jenkins user that will be used to create the Jenkins Jobs.

| +| `--publishCredsId` |

Required

 

The ID of the Jenkins credentials needed to upload LST artifacts to your artifact repository.

| +| `--publishUrl` |

Required

 

The URL of the Maven repository where LST artifacts should be uploaded to.

| +| `--agent` |

Optional

 

The name of the Jenkins agent that will run the pipeline.

| +| `--apiToken` |

Optional

 

The Jenkins apiToken that will be used when authentication is needed in Jenkins (e.g., the creation of Jenkins Jobs).

| +| `--cliVersion` |

Optional

 

The version of the Moderne CLI that should be used when running Jenkins Jobs.

| +| `--defaultBranch` |

Optional

 

If no Git branch is specified for a repository in the CSV file, the Jenkins Job will attempt to checkout this branch when pulling down the code.

| +| `--defaultGradle` |

Optional

 

The name of the Gradle tool that should be used to run Gradle jobs. Specified in the Jenkins Global Tool Configuration page.

| +| `--defaultJdk` |

Optional

 

The name of the JDK tool that should be used to run Java jobs. Specified in the Jenkins Global Tool Configuration page.

| +| `--defaultMaven` |

Optional

 

The name of the Maven tool that should be used to run Maven jobs. Specified in the Jenkins Global Tool Configuration page.

| +| `--deleteSkipped` |

Optional

 

If set to true, whenever a repository in the CSV file has skip set to true, the corresponding Jenkins Job will be deleted.


This is useful if you want to remove specific jobs that are failing, but you also want to preserve the list of repositories that are ingested.

| +| `--downloadCLI` |

Optional

 

Specifies whether or not the Moderne CLI should be downloaded at the beginning of each Jenkins Job run.

| +| `--mavenSettingsConfigFileId` |

Optional

 

The ID of the Jenkins Maven settings config file that will be used to configure Maven builds. Specified in the Jenkins Global Tool Configuration page.

| +| `--folder` |

Optional

 

The Jenkins folder that will store the created jobs. This folder will be created if it does not exist.

| +| `--jenkinsPassword` |

Optional

 

The Jenkins password that will be used when authentication is needed in Jenkins (e.g., the creation of Jenkins Jobs).

Jenkins best practices recommend using an apiToken instead of a password.

| +| `--mirrorUrl` |

Optional

 

For Gradle projects, this can be specified as a Maven repository cache/mirror to check for artifacts before any other repositories are looked at.

| +| `--platform` |

Optional

 

The OS platform for the Jenkins node/agent. The possible options are: windows, linux, or macos.

| +| `--prefix` |

Optional

 

If specified, Jenkins Jobs will only be created for repositories that start with this prefix.

| +| `--scheduledAt` |

Optional

 

The cron schedule that the Jenkins Jobs should follow. By default, Jenkins will execute each job once a day while making sure to space them out so that the system is not overloaded at one particular time.

| + +### Connect GitHub + +The `connect github` command will create a GitHub workflow that builds and publishes LST artifacts to your artifact repository on a regular basis. A workflow can be created for ingesting a single repository (by specifying the `path` parameter) or a workflow can be created for ingesting a mass number of repositories (by specifying the `fromCsv` parameter). + +**If you specify the `path` parameter**: + +This command will create a `moderne-workflow.yml` file in the `.github/workflows` directory at the `path` you specified. This workflow file can then be modified and published to a GitHub repository to set up the workflow for building and publishing LST artifacts for that repository. + +When running this command you will need to ensure that you provide the `publishPwdSecretName`, `publishUrl`, and `publishUserSecretName` parameters. + +For the `publishPwdSecretName` and `publishUserSecretName` parameters, the expectation is that you will create a GitHub secret for each inside of the repository you're wanting to ingest. When running this command, you'd then provide the names of these secrets rather than the secrets themselves (e.g., `--publishPwdSecretName `). + +**If you specify the `fromCsv` parameter**: + +This command will directly commit an ingestion workflow and the necessary files to run it to the GitHub repository you specify. This workflow will iterate over every repository in the CSV and build/publish LST artifacts for each. + +Before running this command, you will need to ensure that you've created a dedicated GitHub repository where all of these files can be uploaded to. + +When running this command, you will need to ensure that you provide the `accessToken`, `publishUrl`, `publishPwdSecretName`, `publishUserSecretName`, `dispatchSecretName`, `repoReadSecretName`, and `repo` parameters. + +For the `publishPwdSecretName`, `publishUserSecretName`, `dispatchSecretName`, and `repoReadSecretName` parameters, the expectation is that you will create a GitHub secret for each inside of the repository you provided to this command. When running this command, you'd then provide the names of these secrets rather than the secrets themselves (e.g., `--publishPwdSecretName `). + +**Example**: + +```sh +mod connect github --accessToken=moderne-github-access-token \ + --dispatchSecretName=dispatchSecretName \ + --fromCsv=/path/to/repos.csv \ + --repoReadSecretName=readSecretName \ + --path=/path/to/project \ + --publishPwdSecretName=repoPasswordName \ + --publishUrl=https://artifact-place.com/artifactory/moderne-ingest \ + --publishUserSecretName=userSecretName + --repo=company-name/repo-name \ +``` + +#### Parameters + +| **Parameter Name** | **Description** | +| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `--accessToken` |

Required (if fromCsv is specified). Optional otherwise.

 

The GitHub access token that will be used to commit files and create workflows.

 

The token requires these permissions: workflow, write:org, and read:org

  • Example: ghp_someAccessToken
| +| `--dispatchSecretName` |

Required (if fromCsv is specified). Optional otherwise.

 

The name of the GitHub secret that contains the access token with write access to the repository specified in the repo parameter. This access token can be used to dispatch GitHub workflows.

 

GitHub secrets can be created inside of of the Security –> Secrets –> Actions section in a GitHub repository.

  • Example: dispatchSecretName
| +| `--fromCsv` |

Either path or fromCsv is required

 

The location of the CSV file containing the list of repositories that should be ingested into Moderne. A GitHub action will build and publish LST artifacts for every repository in this file.

 

Follows the schema of: [repoName,branch,javaVersion,style,buildAction,skip,skipReason].

  • Example: /path/to/repos.csv
| +| `--path` |

Either path or fromCsv is required

 

The local path to the Git repository where a GitHub workflow should be created.

  • Example: /path/to/local/repo
| +| `--publishPwdSecretName` |

Required

 

The name of the GitHub secret that contains the password needed to upload LST artifacts to your artifact repository.

 

GitHub secrets can be created inside of of the Security –> Secrets –> Actions section in a GitHub repository.

  • Example: passwordSecretName
| +| `--publishUrl` |

Required

 

The URL of the Maven formatted artifact repository where LST artifacts should be uploaded to.

  • Default: Will default to the environment variable MODERNE_PUBLISH_URL if one exists.
  • Example: https://some-artifact-repo.com
| +| `--publishUserSecretName` |

Required

 

The name of the GitHub secret that contains the username needed to upload LST artifacts to your artifact repository.

 

GitHub secrets can be created inside of of the Security –> Secrets –> Actions section in a GitHub repository.

  • Example: userSecretName
| +| `--repo` |

Required (if fromCsv is specified). Optional otherwise.

 

The dedicated GitHub repository where the workflows and the CSV file will be committed to. Follows the format of organization/repository.

  • Example: openrewrite/rewrite
| +| `--repoReadSecretName` |

Required (if fromCsv is specified). Optional otherwise.

 

The name of the GitHub secret that contains the access token with read access to each repository in the provided CSV.

 

GitHub secrets can be created inside of of the Security –> Secrets –> Actions section in a GitHub repository.

  • Example: gitSecretName
| +| `--additionalBuildArgs` |

Optional

 

Additional arguments that are added to the Maven or Gradle build command.

  • Example: -Dmaven.antrun.skip=true -pl !packaging/pom.xml
| +| `--apiUrl` |

Optional

 

The base URL for the GitHub REST API. For GitHub enterprise users, this commonly follows the format of http(s)://HOSTNAME/api/v3.

  • Default: https://api.github.com
| +| `--branch` |

Optional

 

The branch of the GitHub repository specified in the repo parameter where the generated workflow files should be committed to.

  • Default: main
| +| `--cliVersion` |

Optional

 

The version of the Moderne CLI that should be used when running the ingestion workflow. Follows standard semantic versioning with a v in front of the version number.

  • Example: v0.0.50
| +| `--javaVersion` |

Optional

 

The Java version needed to compile and run the repository that is indicated in the path parameter. Can be any major version (8, 11, 17).

  • Default: 11
|