Command-line reference

connect_manager

connect_manager is a CLI for the Confluent Kafka Connect REST API, it makes it easy to manage connectors.

connect_manager [OPTIONS] COMMAND [ARGS]...

Options

--broker <broker_url>

Kafka broker. Alternatively set via $BROKER env var. [default: confluent-kafka-cp-kafka-headless:9092]

--kafka-connect <kafka_connect_url>

Kafka Connect URL. Alternatively set via $KAFKA_CONNECT env var. [default: http://confluent-kafka-cp-kafka-connect:8083]

--version

Show the version and exit.

Environment variables

BROKER
Provide a default for --broker
KAFKA_CONNECT
Provide a default for --kafka-connect

create

Create a new connector. Each create subcommand manages a different connector.

connect_manager create [OPTIONS] COMMAND [ARGS]...

influxdb-sink

Create an instance of the Stream Reactor InfluxDB Sink Connector.

The TOPICS argument specifies the list of topics to write to InfluxDB. If not provided, topics are discovered from Kafka. In this case, use the –filter option to regex the topics you want to write to InfluxDB.

connect_manager create influxdb-sink [OPTIONS] [TOPICS]...

Options

--name <name>

Name of the connector to create. [default: influxdb-sink]

--influxdb_url <influxdb_url>

InfluxDB connection URL. Alternatively set via $INFLUXDB env var. [default: https://localhost:8086]

-d, --database <database>

InfluxDB database name. The database must exist at InfluxDB. [required]

-t, --tasks <tasks>

Number of Kafka Connect tasks. [default: 1]

-u, --username <username>

InfluxDB username. Alternatively set via $INFLUXDB_USER env var.Use ‘-‘ for unauthenticated users.

-p, --password <password>

InfluxDB password. Alternatively set via $INFLUXDB_PASSWORD env var.

-f, --filter <filter_regex>

Regex for selecting topics.

--dry-run

Show the InfluxDB Sink Connector configuration but does not create the connector.

--auto-update

Check for new Kafka topics and updates the connector configuration.–auto-update does not take effect if –dry-run is used.

--check-interval <check_interval>

The interval, in milliseconds, to check for new topics and updatethe connector.

--blacklist <blacklist>

Blacklist problematic topics.

--timestamp <timestamp>

Timestamp to use when recording the message in InfluxDB. [default: sys_time()]

--error-policy <error_policy>

Specifies the action to be taken if an error occurs while inserting the data. There are three available options, NOOP, the error is swallowed, THROW, the error is allowed to propagate and RETRY. For RETRY the Kafka message is redelivered up to a maximum number of times specified by the –max-retries option. The retry interval is specified by the –retry-interval option. [default: THROW]

Options:NOOP|THROW|RETRY
--max-retries <max_retries>

The maximum number of times a message is retried. Only valid when the –error-policy is set to RETRY. [default: 10]

--retry-interval <retry_interval>

The interval, in milliseconds between retries. Only valid when the –error-policy is set to RETRY. [default: 60000]

Arguments

TOPICS

Optional argument(s)

Environment variables

INFLUXDB
Provide a default for --influxdb_url
INFLUXDB_USER
Provide a default for --username
INFLUXDB_PASSWORD
Provide a default for --password

replicator

Create an instance of the Replicator Connector.

The TOPICS argument specifies the list of topics to replicate. If provided, they are added to the whitelist. If not, only topics matching the –filter regex option will be replicated. Whitelisted topics do not have to match the regex.

connect_manager create replicator [OPTIONS] [TOPICS]...

Options

--name <name>

Name of the connector to create. [default: replicator]

--src-kafka <src_kafka>

A list of host and port pairs to use for establishing the initial connection to the source Kafka cluster. This list must be in the form host1:port1,host2:port2,.... [required]

--dest-kafka <dest_kafka>

A list of host and port pairs to use for establishing the initial connection to the destination Kafka cluster. This list must be in the form host1:port1,host2:port2,.... [required]

--topic-rename-format <topic_rename_format>

A format string for the topic name in the destination cluster, which may contain ${topic} as a placeholder for the originating topic name, e.g. summit_${topic}. [default: ${topic}]

-t, --tasks <tasks>

Number of Kafka Connect tasks. [default: 1]

--schema-registry-topic <schema_registry_topic>

The topic that acts as the durable log for the schema registry. [default: _schemas]

--schema-registry-url <schema_registry_url>

Comma-separated list of URLs for schema registry instances that can be used to register or look up schemas. If the replicator is configured in the destination cluster this should be the URL for the destination schema registry which configured in IMPORT mode. [required]

--group-id <group_id>

If the replicator needs to run on a separate Connect cluster, the group.id property specifies which workers to use.

-f, --filter <filter_regex>

Regex of topics to replicate to the destination cluster. [required]

--dry-run

Show the InfluxDB Sink Connector configuration but does not create the connector.

--blacklist <blacklist>

Topics to exclude from replication. Note that Kafka internal topics are not replicated by default.

--check-interval <check_interval>

How often to poll the source cluster for new topics matching the –filer regex, in milliseconds. [default: 120000]

Arguments

TOPICS

Optional argument(s)

delete

Delete a connector.

Halt all tasks and delete the connector configuration.

connect_manager delete [OPTIONS] NAME

Arguments

NAME

Required argument

help

Show help for any command.

connect_manager help [OPTIONS] [TOPIC]

Arguments

TOPIC

Optional argument

info

Get information about the connector.

connect_manager info [OPTIONS] NAME

Arguments

NAME

Required argument

list

Get a list of active connectors.

connect_manager list [OPTIONS]

pause

Pause the connector and its tasks.

connect_manager pause [OPTIONS] NAME

Arguments

NAME

Required argument

restart

Restart a connector and its tasks.

connect_manager restart [OPTIONS] NAME

Arguments

NAME

Required argument

resume

Resume a paused connector.

connect_manager resume [OPTIONS] NAME

Arguments

NAME

Required argument

status

Get current status of the connector.

Whether it is running, failed or paused, which worker it is assigned to, error information if it has failed, and the state of all its tasks.

connect_manager status [OPTIONS] NAME

Arguments

NAME

Required argument