From 762c5d7151188892be3bd5f755a087175517b0ce Mon Sep 17 00:00:00 2001 From: Joe Bellus Date: Sun, 25 Sep 2022 03:23:44 -0400 Subject: [PATCH 01/11] Added ci --- .drone.yml | 6 + .hugo_build.lock | 0 config.toml | 2 +- content/articles/command.md | 51 +++---- content/articles/config.md | 213 ++++++++++++---------------- content/articles/getting-started.md | 60 ++++---- themes/5Sigma/layouts/index.html | 61 +------- 7 files changed, 146 insertions(+), 247 deletions(-) create mode 100644 .drone.yml create mode 100644 .hugo_build.lock diff --git a/.drone.yml b/.drone.yml new file mode 100644 index 0000000..1a4af68 --- /dev/null +++ b/.drone.yml @@ -0,0 +1,6 @@ +steps: +- name: build + image: plugins/hugo + settings: + hugo_version: 0.104.0 + validate: true diff --git a/.hugo_build.lock b/.hugo_build.lock new file mode 100644 index 0000000..e69de29 diff --git a/config.toml b/config.toml index 11f30ab..3aa5ada 100644 --- a/config.toml +++ b/config.toml @@ -4,4 +4,4 @@ title = "Conductor: Development Stack Launcher" theme="5Sigma" pygmentsUseClasses=true pygmentsCodeFences=true -publishDir="docs" \ No newline at end of file +publishDir="dist" \ No newline at end of file diff --git a/content/articles/command.md b/content/articles/command.md index f19710a..b48595d 100644 --- a/content/articles/command.md +++ b/content/articles/command.md @@ -1,66 +1,55 @@ --- title: "Command Reference" -date: 2020-07-29T22:45:07-04:00 +date: 2022-09-10T22:45:07-04:00 description: "Full command usage" draft: false weight: 1 -version: "0.3.2" +version: "2.0.0" --- # Comamnd Reference -## Launching a stack +The command can be used to launch any number of configured objects using a space separated list. -The full stack listed in a configuration can be run by running the bare conductor command: +## Listing configured objects + +Running the command with no arguments will list all configured groups, components, and tasks that are configured in the configuration file. ```sh -conductor +conductor ``` -The run subcommand without arguments can also be used: +## Launching a specific item + +Launchign a group ```sh -conductor run +conductor groupname ``` -## Launching via tags - -To run a group of components without launching the entire stack specify tags in the component configuration. One or more tags can be specified to run only components that have at least one of the specified tags. +Launching a specific component ```sh -conductor run --tags web,frontend +conductor componentname ``` -## Launching by name - -The name of a component can be used as a subcommand to launch that component by itself: +Launching a specific root level task ```sh -conductor my-component +conductor taskname ``` -Multiple components can be ran by specifying multiple component subcommands: +Launching a compoennt level task ```sh -conductor server frontend metrics +conductor componentname:taskname ``` -## Launching groups +## Launching Multiple Specific Items -Groups of components can be defined in the conductor.yml and launched by name +Multiple objects can be ran with a space separated list. Groups, components, and tasks can be used interchangably. ```sh -conductor my_group +conductor mygroup mytask mycomponent mycomponent:subtask ``` - -## Setup a stack - -Given a configuration file you can clone and initialize an entire stack. This can be useful for bootstrapping a new computer/developer. -From a folder with the conductor.yaml run: - -```sh -conductor setup -``` - -Each component will be cloned into subfolders named after their component name (unless path is set) and their init commands will be run. \ No newline at end of file diff --git a/content/articles/config.md b/content/articles/config.md index 80b7626..f82b211 100644 --- a/content/articles/config.md +++ b/content/articles/config.md @@ -1,163 +1,124 @@ --- title: "Configuration File Reference" -date: 2020-07-29T20:42:50-04:00 +date: 2022-09-10T20:42:50-04:00 description: "Conductor config file reference" -version: "0.3.2" +version: "2.0.0" weight: 1 --- - # Configuration file Conductor will look in the current directory or any parent directory for configuration file named `conductor.yaml`. In general you can place this configuration directory in or above any place you might want to run the stack. -## Manually specifying the configuration file -If the configuration file is located outside the current directory structure you can specify it with the -c flag. +## Group configuration -```sh -conductor -c /path/to/config/conductor.yaml -``` - -## Project configuration - -The top level of the configuration has various configurational options for the entire stack. +Groups can be defined to launch multiple components at a time. This is the primary use case for Conductor and is where development flows are defined. +Generally, you define all the separate compoennts required to setup a development environment and launch them via a group. ```yml -name: MyProject -services: -- name: mysql -- name: rabbit groups: -- name: group1 - components: - - component1 - - component2 -- name: group2 - components: - - component1 - - component3 + group1: + components: + - component1 + - component2 + group2: + components: + - component1 + - component3 components: # ... ``` -- **name** - An arbitrary name for the project -- **services** - An array of external services that can be launched by components. Currently only docker containers are supported. - - **name** - The name of the docker container to launch -- **groups** - An array of groups can be defined to execute multiple components together - - **name** - An arbitrary name for the group - - **components** - An array of component names that should be launched as a group - +The key for the group item is the name of the group. +### Group subfields: +- **components** - An array of component names that should be launched as a group ## Component configuration -Components are indvidual applications that are run as part of the stack. All, some, or one of the compoennts can be launched. +Components are indvidual applications that are run as part of the stack. Components can be launched individually or via groups. ### Example component configuration: ```yaml -- name: api - color: Red - path: backend/api-gateway - tags: - - api - - web - env: - COLORS: 1 - NPM_ENV: debug - start: - command: npm - args: - - start - init: - - command: npm - args: - - install - repo: - https://github.com/me/my-project.git +components: + api: + path: backend/api-gateway + env: + COLORS: 1 + NPM_ENV: debug + command: npm start + before: npm-install + tasks: + npm-install: + path: backend/api-gateway + command: npm install ``` -* **name** - The name for the component. This name is used in the log output and can also be specified as a sub-command to run the -component itself. -* **path** - The working path relative to the configuration file. If not specified the path is assuemd to be a subfolder with the component name. -* **env** - A set of environment variables that are set before any commands are run. -* **color** - The component will use this color in the log output. Valid color values are: Yellow, Blue, Green, Red, and Purple. -* **tags** - A list of tags to identify the component. These can be used to execute groups of components using the --tags flag. -* **start** - A command block that is executed when the component is ran -* **init** - A list of command blocks that are ran when the component is initialized -* **repo** - The repository url for the component. When running init all components are cloned into subfolders if they have repositories specified and their init commands are ran. -* **retry** - Specifies whether the command should be rerun if it exits. This is true by default. Setting this to false will cause the command to be executed. This does not apply to init commands. -* **delay** - A delay in seconds to wait before executing this command. This is useful if it needs to wait for another component to spin up. -* **default** - If set to false the component will not be ran when the bare `conductor` or `conductor run` is executed. +The key for the component item is the name of the component. -### Command block reference - * **command** - the name of the command to run - * **args** - A list of arguments to pass to the command - * **env** - A list of environment variables to set specific to this command - * **dir** - An optional working directory override if the command should be ran somewhere other than the component path +component itself. +* **path** - The working path relative to the configuration file. By default the path is "." +* **env** - A set of environment variables that are set before any commands are run. +* **command** - The command that should be executed when the component is launched. This command can be a shell command. Multiple commands can be specified as an array or a single command can be specified as a string +* **keep_alive** - Specifies whether the command should be rerun if it exits. This is true by default. +* **retry_delay** - A delay in seconds to wait before launching the command after it exits. +* **tasks** - A list of task definitions that are scoped within this compoennt. Tasks can be specified under a component for organizational reasons. These components become _namespaced_ and can be executed directly by using its qualified name: component:task. See the task definitions for information about these blocks. + +## Task configuration + +Tasks are similiar to components, except they are meant as single run utility commands. These can be used to setup the environment, or as pure utility functions like build pipelines and data configuration. + +The key for the task item is the name of the task. ## Full example config: ```yaml -name: My Web Application -services: -- name: mysql groups: -- name: full - components: - - frontend - - backend - + dev: + components: + - frontend + - backend + prod: + components: + - frontend + - backend-prod components: -- name: frontend - env: - FORCE_COLOR: 1 - color: Blue - tags: - - core - start: - command: npm - args: - - start - init: - - command: npm - args: - - install -- name: backend - services: - - mysql - color: Purple - tags: - - core - start: - - command: mix - args: - - phx.server - init: - - command: mix - args: - - deps.get - - command: mix - args: - - compile -- name: production-backend - default: false - env: - MIX_ENV: production - color: Purple - tags: - - core - start: - - command: mix - args: - - phx.server - init: - - command: mix - args: - - deps.get - - command: mix - args: - - compile -``` \ No newline at end of file + frontend: + env: + FORCE_COLOR: 1 + command: npm start + before: setup + tasks: + setup: + command: npm install + backend: + command: mix phx.server + before: backend:setup + tasks: + setup: + before: + - backend:compile + - backend:deps + compile: + command: mix compile + deps: + command: mix deps.get + backend-prod: + env: + MIX_ENV: production + command: mix phx.server + before: backend:setup +tasks: + reset: + env: + MIX_ENV: production + command: + - mix ecto.reset + - mix ecto.migrate + - mix ecto.seed + + +``` diff --git a/content/articles/getting-started.md b/content/articles/getting-started.md index 1d7635f..e329908 100644 --- a/content/articles/getting-started.md +++ b/content/articles/getting-started.md @@ -1,24 +1,12 @@ --- title: "Getting Started" -date: 2020-06-28T11:29:09-04:00 +date: 2022-09-10T11:29:09-04:00 description: "A quick setup guide to get a server running" draft: false -version: "0.3.2" +version: "2.0.0" --- -# Installation - Binaries can be downloaded for [Windows](https://github.com/5Sigma/conductor/releases/latest/download/conductor-windows.zip), [Linux](https://github.com/5Sigma/conductor/releases/latest/download/conductor-linux.tar.gz), - or [MacOS](https://github.com/5Sigma/conductor/releases/latest/download/conductor-darwin.zip) - -## Homebrew - -On MacOS Conductor can be installed via Homebrew -```sh -brew tap 5sigma/tap -brew install conductor -``` - # Setting up a project To setup a project for conductor you need to create a `conductor.yaml` somewhere in your project tree. @@ -41,35 +29,43 @@ in the root `/project/` folder so it is accessible from any of the component pro For more information see the full Config documentation {{< /infobox >}} -An example of a minimal configuration file could look like this: +An example of a configuration file could look like this: ```yaml +groups: + dev: + components: + - frontend + - backend + - support components: -- name: frontend - start: - command: npm - args: - - start -- name: backend - start: - command: npm - args: - - start -- name: support - start: - command: python - args: - - main.py + frontend: + env: + API_URL: http://localhost:4000 + comamnd: npm start + backend: + env: + DATABSE_URL: mysql://devserver/app-dev + HTTP_PORT: 4000 + command: cargo run + support: + command: python main.py ``` Now from the project folder or any folder below it we can launch all 3 components using: ```sh -conductor run +conductor dev ``` -Alternatively, we can run a single component by name: +Running a single component: ```sh conductor backend ``` + +Running multiple specific components: + +```sh +conductor frnotend backend +``` diff --git a/themes/5Sigma/layouts/index.html b/themes/5Sigma/layouts/index.html index 45c0e88..2db7ec9 100644 --- a/themes/5Sigma/layouts/index.html +++ b/themes/5Sigma/layouts/index.html @@ -18,68 +18,15 @@

-
-
-
- - Windows -
-
- - Linux -
-
- - MacOS -
- - - Source Code - -
-
-
+
Binary Download
- Download - -
-
-
- Binary Download -
- - - Donwload - -
-
-
- Hombrew Installation -
-
-
- 
-            brew tap 5sigma/tap
-            brew install conductor
-
-
-
- Or -
-
- Binary Download -
- - - - Download + Download +

Conductor currently only support Linux based systems

@@ -139,4 +86,4 @@ - \ No newline at end of file + -- 2.40.1 From 49a9b15fe36181e34fb8c5f5f09e5de408be8801 Mon Sep 17 00:00:00 2001 From: Joe Bellus Date: Sun, 25 Sep 2022 03:25:11 -0400 Subject: [PATCH 02/11] Added kind to ci --- .drone.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.drone.yml b/.drone.yml index 1a4af68..c8ac568 100644 --- a/.drone.yml +++ b/.drone.yml @@ -1,3 +1,5 @@ +kind: pipeline + steps: - name: build image: plugins/hugo -- 2.40.1 From 8d1b21eb1d2cb3d5967f29f62edefa56643f294b Mon Sep 17 00:00:00 2001 From: Joe Bellus Date: Sun, 25 Sep 2022 03:26:42 -0400 Subject: [PATCH 03/11] Changed hugo version to 0.55 --- .drone.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.drone.yml b/.drone.yml index c8ac568..37cf194 100644 --- a/.drone.yml +++ b/.drone.yml @@ -4,5 +4,5 @@ steps: - name: build image: plugins/hugo settings: - hugo_version: 0.104.0 + hugo_version: 0.55 validate: true -- 2.40.1 From b2fbf96f976ebd81ad656ab9173981493f21005e Mon Sep 17 00:00:00 2001 From: Joe Bellus Date: Sun, 25 Sep 2022 03:28:24 -0400 Subject: [PATCH 04/11] Removed hugo validate --- .drone.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/.drone.yml b/.drone.yml index 37cf194..c566d52 100644 --- a/.drone.yml +++ b/.drone.yml @@ -5,4 +5,3 @@ steps: image: plugins/hugo settings: hugo_version: 0.55 - validate: true -- 2.40.1 From c6f13745e990d55f82987ea4de0bc79d45a9956a Mon Sep 17 00:00:00 2001 From: Joe Bellus Date: Sun, 25 Sep 2022 03:34:38 -0400 Subject: [PATCH 05/11] Using standard images --- .drone.yml | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/.drone.yml b/.drone.yml index c566d52..67c2225 100644 --- a/.drone.yml +++ b/.drone.yml @@ -2,6 +2,22 @@ kind: pipeline steps: - name: build - image: plugins/hugo - settings: - hugo_version: 0.55 + image: hugo:latest + volumes: + - name: dist + path: dist/ + commands: + - hugo +- name: deploy + image: amazon/aws-cli + environment: + AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY + AWS_DEFAULT_REGION: us-east-1 + AWS_SECRET_ACCESS_KEY: $AWS_SECRET + volumes: + - name: dist + path: dist/ + commands: + - hugo + - aws s3 sync ./dist s3://conductor.5sigma.io --delete; + -- 2.40.1 From 006f3a171dae64121aa02a335dd61c1c4062510e Mon Sep 17 00:00:00 2001 From: Joe Bellus Date: Sun, 25 Sep 2022 03:36:41 -0400 Subject: [PATCH 06/11] Switched hugo image --- .drone.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.drone.yml b/.drone.yml index 67c2225..7d5f640 100644 --- a/.drone.yml +++ b/.drone.yml @@ -2,7 +2,7 @@ kind: pipeline steps: - name: build - image: hugo:latest + image: klakegg/hugo volumes: - name: dist path: dist/ -- 2.40.1 From c9c307dec5b48944efa0eaeae71afa9023c91744 Mon Sep 17 00:00:00 2001 From: Joe Bellus Date: Sun, 25 Sep 2022 03:37:40 -0400 Subject: [PATCH 07/11] Removed hugo from deploy step --- .drone.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/.drone.yml b/.drone.yml index 7d5f640..2021a14 100644 --- a/.drone.yml +++ b/.drone.yml @@ -18,6 +18,5 @@ steps: - name: dist path: dist/ commands: - - hugo - aws s3 sync ./dist s3://conductor.5sigma.io --delete; -- 2.40.1 From cf02e025b93c0bfd6e2303cf00a1b38fc229dfbd Mon Sep 17 00:00:00 2001 From: Joe Bellus Date: Sun, 25 Sep 2022 03:42:15 -0400 Subject: [PATCH 08/11] updated aws env from secret --- .drone.yml | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/.drone.yml b/.drone.yml index 2021a14..27770a5 100644 --- a/.drone.yml +++ b/.drone.yml @@ -11,9 +11,11 @@ steps: - name: deploy image: amazon/aws-cli environment: - AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY + AWS_ACCESS_KEY_ID: + from_secret: AWS_ACCESS_KEY + AWS_SECRET_ACCESS_KEY: + from_secret; AWS_SECRET AWS_DEFAULT_REGION: us-east-1 - AWS_SECRET_ACCESS_KEY: $AWS_SECRET volumes: - name: dist path: dist/ -- 2.40.1 From 2f48179dd46805fc394770f5a47f9acf3b4f99a1 Mon Sep 17 00:00:00 2001 From: Joe Bellus Date: Sun, 25 Sep 2022 03:43:19 -0400 Subject: [PATCH 09/11] Fixed typo --- .drone.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.drone.yml b/.drone.yml index 27770a5..4fdd2b9 100644 --- a/.drone.yml +++ b/.drone.yml @@ -14,7 +14,7 @@ steps: AWS_ACCESS_KEY_ID: from_secret: AWS_ACCESS_KEY AWS_SECRET_ACCESS_KEY: - from_secret; AWS_SECRET + from_secret: AWS_SECRET AWS_DEFAULT_REGION: us-east-1 volumes: - name: dist -- 2.40.1 From 870313c7877f86aeed0e8e655b9222740153271f Mon Sep 17 00:00:00 2001 From: Joe Bellus Date: Sun, 25 Sep 2022 03:44:17 -0400 Subject: [PATCH 10/11] Added promotion to deploy --- .drone.yml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/.drone.yml b/.drone.yml index 4fdd2b9..a9f4a3c 100644 --- a/.drone.yml +++ b/.drone.yml @@ -21,4 +21,11 @@ steps: path: dist/ commands: - aws s3 sync ./dist s3://conductor.5sigma.io --delete; + when: + event: + - promote + target: + - staging + - production + -- 2.40.1 From 164a1ec83c3e4ffc5cde190541f85e9c0bcbcfbd Mon Sep 17 00:00:00 2001 From: Joe Bellus Date: Sun, 25 Sep 2022 13:49:26 -0400 Subject: [PATCH 11/11] Updated docs for refactor --- .gitignore | 2 +- content/articles/config.md | 23 +++++++++++++++++++++-- themes/5Sigma/layouts/index.html | 9 ++++++++- 3 files changed, 30 insertions(+), 4 deletions(-) diff --git a/.gitignore b/.gitignore index 96c0ecc..7773828 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1 @@ -docs/ \ No newline at end of file +dist/ \ No newline at end of file diff --git a/content/articles/config.md b/content/articles/config.md index f82b211..f6fc6ee 100644 --- a/content/articles/config.md +++ b/content/articles/config.md @@ -35,6 +35,7 @@ The key for the group item is the name of the group. ### Group subfields: - **components** - An array of component names that should be launched as a group +- **descirption** - A description that is displayed in the task list ## Component configuration @@ -58,21 +59,39 @@ components: The key for the component item is the name of the component. -component itself. +### Component subfields: * **path** - The working path relative to the configuration file. By default the path is "." * **env** - A set of environment variables that are set before any commands are run. * **command** - The command that should be executed when the component is launched. This command can be a shell command. Multiple commands can be specified as an array or a single command can be specified as a string * **keep_alive** - Specifies whether the command should be rerun if it exits. This is true by default. * **retry_delay** - A delay in seconds to wait before launching the command after it exits. * **tasks** - A list of task definitions that are scoped within this compoennt. Tasks can be specified under a component for organizational reasons. These components become _namespaced_ and can be executed directly by using its qualified name: component:task. See the task definitions for information about these blocks. +* **descirption** - A description that is displayed in the task list ## Task configuration Tasks are similiar to components, except they are meant as single run utility commands. These can be used to setup the environment, or as pure utility functions like build pipelines and data configuration. - The key for the task item is the name of the task. +### Example task configuration: +```yaml +tasks: + build: + env: + MODE: production + path: backend/ + command: cargo build --release + description: Backend release build +``` + +### Task subfields: +* **path** - The working path relative to the configuration file. By default the path is "." +* **env** - A set of environment variables that are set before any commands are run. +* **command** - The command that should be executed when the component is launched. This command can be a shell command. Multiple commands can be specified as an array or a single command can be specified as a string +* **descirption** - A description that is displayed in the task list + + ## Full example config: ```yaml diff --git a/themes/5Sigma/layouts/index.html b/themes/5Sigma/layouts/index.html index 2db7ec9..45d85d4 100644 --- a/themes/5Sigma/layouts/index.html +++ b/themes/5Sigma/layouts/index.html @@ -23,11 +23,18 @@ Binary Download + href="https://objects.5sigma.io/public/conductor.tar.gz"> Download

Conductor currently only support Linux based systems

+
+
+ 
+curl https://objects.5sigma.io/public/conductor.tar.gz | tar -xz 
+./conductor
+
+
-- 2.40.1