Installing and running

Homebrew

The recommended way of running Conduit on a local machine is using the latest version of Conduit available on Homebrew which you can instally by running the following command:

brew install conduit

Run

Once installed, you can start Conduit by running:

conduit

Using a pre-built binary

Below we'll walk through getting Conduit installed and running on your system.

Download

First, download the latest Conduit release for your platform.

For an ARM based back Mac for example we'd grab the conduit_0.11.1_Darwin_arm64.tar.gz archive.

unzip the archive

tar xzvf conduit_0.11.1_Darwin_arm64.tar.gz

Run

In the directory in which you expanded the archive. Execute the Conduit binary:

./conduit

Docker

We distribute Docker images to ease the use of Conduit in a containerized environment. To see details about how our images are built please feel free to take a look at the Dockerfile here. Conduit's Docker images are hosted on GitHub.

Run

After you install Docker, you can pull the Conduit image by running the following command:

docker pull conduit.docker.scarf.sh/conduitio/conduit

Then, you can run Conduit by running:

docker run -it -p 8080:8080 conduit.docker.scarf.sh/conduitio/conduit

The command aboves runs Conduit with no pipelines. To run Conduit with a custom pipeline file you can mount it as a volume into the container:

docker run -it -p 8080:8080 \
  -v /path/to/pipeline.yaml:/app/pipelines/pipeline.yaml \
  conduit.docker.scarf.sh/conduitio/conduit

tip

If you're using Docker, check out Watchtower. After we push an update, Watchtower will keep your Conduit container updated by gracefully shutting down your existing container and restarting it with the same options used when initially deployed.

Building from source

Before you can build Conduit from source, you need to have Go 1.22 or higher installed. The Go website has comprehensive instructions on how to get the language installed on your machine. Once installed, you can follow the rest of the instructions.

1. Start by downloading the source code from the latest stable release on the Conduit Releases Page. Alternatively, you can run this command to automatically download the latest stable source to your current directory:

$ TAG=v0.11.1; curl -o conduit.tar.gz -L https://github.com/ConduitIO/conduit/archive/refs/tags/$TAG.tar.gz

A file called conduit.tgz will be in your current directory. The next step is to expand the source:

$ tar zxvf conduit.tgz

Then change directories to the appropriate folder. Keep in mind that the folder name might be different between releases since it's tied to the latest git sha for the commit.

$ cd conduit-0.11.1

Now build the project:

$ make

If you want to build Conduit without the UI, instead of running make, you can run the following command:

$ make build-server

You will have a new binary built for your architecture and machine. All that's left is to run it!

./conduit

You should now be able to interact with the Conduit UI and HTTP API on port 8080:

             ....
         .::::::::::.
       .:::::‘‘‘‘:::::.
      .::::        ::::.
 .::::::::          ::::::::.
 `::::::::          ::::::::‘
      `::::        ::::‘
       `:::::....:::::‘
         `::::::::::‘        Conduit v0.11.1 linux/amd64
             ‘‘‘‘
2024-02-20T21:37:45+00:00 INF All 0 tables opened in 0s component=badger.DB
2024-02-20T21:37:45+00:00 INF Discard stats nextEmptySlot: 0 component=badger.DB
2024-02-20T21:37:45+00:00 INF Set nextTxnTs to 0 component=badger.DB
2024-02-20T21:37:45+00:00 INF builtin plugins initialized component=builtin.Registry count=6
2024-02-20T21:37:45+00:00 WRN could not read plugin directory error="open /app/connectors: no such file or directory" component=standalone.Registry
2024-02-20T21:37:45+00:00 INF standalone plugins initialized component=standalone.Registry count=0 plugin_path=/app/connectors
2024-02-20T21:37:45+00:00 INF processors initialized component=processor.Service count=0
2024-02-20T21:37:45+00:00 INF connectors initialized component=connector.Service count=0
2024-02-20T21:37:45+00:00 INF pipelines initialized component=pipeline.Service count=0
2024-02-20T21:37:45+00:00 INF pipeline configs provisioned component=provisioning.Service created=[] deleted=[] pipelines_path=./pipelines
2024-02-20T21:37:45+00:00 INF grpc server started address=[::]:8084
2024-02-20T21:37:45+00:00 INF http server started address=[::]:8080
2024-02-20T21:37:45+00:00 INF
2024-02-20T21:37:45+00:00 INF click here to navigate to Conduit UI: http://localhost:8080/ui
2024-02-20T21:37:45+00:00 INF click here to navigate to explore the HTTP API: http://localhost:8080/openapi
2024-02-20T21:37:45+00:00 INF

Next Steps

Now that you have Conduit installed it's time to explore some other topics.

• Pipelines
• Connectors
• Processors