Connecting Clickhouse to pupular OLTP DBMSs

Connecting Clickhouse to pupular OLTP DBMSs

April 20, 2023
Database

How to Connect Clickhouse to PostgreSQL, MySQL, Oracle, SQL-Server, MongoDB.

This post shows a step-by-step tutorial for connecting ClickHouse to PostgreSQL, MySQL and MongoDB via the ClickHouse storage engine and connecting Oracle and SQL-Server via the JDBC-bridge(the official JDBC-bridge has some limitations, check this fork as a workaround)

Connecting Clickhouse to PostgreSQL #

1. Setting up PostgreSQL #

  1. Login to postgres
psql -U postgres
  1. Create a new database in PostgreSQL:
CREATE DATABASE db_in_psg;
  1. Create two new table:
CREATE TABLE table1 (
    id         integer primary key,
    column1    varchar(10)
);

CREATE TABLE table2 (
    txt        varchar(10) primary key,
    column2    varchar(10)
);

CREATE VIEW view1 AS SELECT * from table1 LEFT JOIN table2 ON table1.column1 = table2.txt;
  1. Add a few rows for testing
INSERT INTO table1
  (id, column1)
VALUES
  (1, 'abc'),
  (2, 'def');

INSERT INTO table2
  (txt, column2)
VALUES
  ('abc', '1'),
  ('def', '2');  

2. Define a Table in ClickHouse #

  1. Login to the clickhouse console
clickhouse-client --user default --password ClickHouse123!
  1. Create a new database
CREATE DATABASE db_in_ch;
  1. Create a table that use the PostgreSQL
use db_in_ch;
CREATE TABLE table1
(
    id UInt64,
    column1 String
)
ENGINE = PostgreSQL('postgres:5432', 'db_in_psg', 'table1', 'postgres', 'password');

CREATE TABLE table2
(
    txt String,
    column2 String
)
ENGINE = PostgreSQL('postgres:5432', 'db_in_psg', 'table2', 'postgres', 'password');

CREATE TABLE view1
(
    id UInt64,
    column1 String,
    txt String,
    column2 String
)
ENGINE = PostgreSQL('postgres:5432', 'db_in_psg', 'view1', 'postgres', 'password');

3 Test the Integration #

  1. In ClickHouse, view initial rows of table1:
SELECT * FROM table1
SELECT *
FROM table1

Query id: ba7eea35-9d42-4462-9ff5-d6fd863f327c

┌─id─┬─column1─┐
│  1 │ abc     │
│  2 │ def     │
└────┴─────────┘
  1. Back in PostgreSQL, add a couple of rows to the table1:
INSERT INTO table1
  (id, column1)
VALUES
  (3, 'ghi'),
  (4, 'jkl');

2 rows in set. Elapsed: 0.006 sec.
  1. Those two new rows should appear in your ClickHouse table1:
SELECT *
FROM table1

Query id: 35005ddc-2987-414b-9bd6-5ab67c542a91

┌─id─┬─column1─┐
│  1 │ abc     │
│  2 │ def     │
│  3 │ ghi     │
│  4 │ jkl     │
└────┴─────────┘

4 rows in set. Elapsed: 0.006 sec.
  1. Check rows of table2
SELECT *
FROM table2

Query id: 325e41d5-314b-4a05-a114-4c53cd35c42b

┌─txt─┬─column2─┐
│ abc │ 1       │
│ def │ 2       │
└─────┴─────────┘

2 rows in set. Elapsed: 0.014 sec.
  1. Check rows of view1
SELECT *
FROM view1

Query id: 2feebb31-1ad2-488a-95ce-50ae158b23ca

┌─id─┬─column1─┬─txt─┬─column2─┐
│  1 │ abc     │ abc │ 1       │
│  2 │ def     │ def │ 2       │
│  3 │ ghi     │     │         │
│  4 │ jkl     │     │         │
└────┴─────────┴─────┴─────────┘

4 rows in set. Elapsed: 0.019 sec.
  1. Back in PostgreSQL, add a couple of rows to table2
INSERT INTO table2
  (txt, column2)
VALUES
  ('ghi', '3'),
  ('jkl', '4'); 
  1. These two new rows should appear in ClickHouse table2:
SELECT *
FROM table2

Query id: 27feca52-317a-4348-80d2-8363677c40ea

┌─txt─┬─column2─┐
│ abc │ 1       │
│ def │ 2       │
│ ghi │ 3       │
│ jkl │ 4       │
└─────┴─────────┘

4 rows in set. Elapsed: 0.002 sec.
  1. These two new rows also appear in ClickHouse view1:
SELECT *
FROM view1

Query id: 1b55441b-7d52-4cfd-88d6-1d6cb5c5b367

┌─id─┬─column1─┬─txt─┬─column2─┐
│  1 │ abc     │ abc │ 1       │
│  2 │ def     │ def │ 2       │
│  3 │ ghi     │ ghi │ 3       │
│  4 │ jkl     │ jkl │ 4       │
└────┴─────────┴─────┴─────────┘

4 rows in set. Elapsed: 0.007 sec.

Connecting Clickhouse to MySQL #

1. Setting up MySQL #

  1. Login to mysql
mysql -uroot -p
  1. Create a new database in MySQL:
CREATE DATABASE db_in_mysql;
use db_in_msyql;
  1. Create two new table:
CREATE TABLE table1 (
    id         integer primary key,
    column1    varchar(10)
);

CREATE TABLE table2 (
    txt        varchar(10) primary key,
    column2    varchar(10)
);

CREATE VIEW view1 AS SELECT * from table1 LEFT JOIN table2 ON table1.column1 = table2.txt;
  1. Add a few rows for testing
INSERT INTO table1
  (id, column1)
VALUES
  (1, 'abc'),
  (2, 'def');

INSERT INTO table2
  (txt, column2)
VALUES
  ('abc', '1'),
  ('def', '2');  

2. Define a Table in ClickHouse #

  1. Login to the clickhouse console
clickhouse-client --user default --password ClickHouse123!
  1. Create a new database
CREATE DATABASE mysql_in_ch;
use mysql_in_ch;
  1. Create a table that use the MySQL
CREATE TABLE table1
(
    id UInt64,
    column1 String
)
ENGINE = MySQL('mysql:3306', 'db_in_mysql', 'table1', 'root', 'Tes9ting');

CREATE TABLE table2
(
    txt String,
    column2 String
)
ENGINE = MySQL('mysql:3306', 'db_in_mysql', 'table2', 'root', 'Tes9ting');

CREATE TABLE view1
(
    id UInt64,
    column1 String,
    txt String,
    column2 String
)
ENGINE = MySQL('mysql:3306', 'db_in_mysql', 'view1', 'root', 'Tes9ting');

3 Test the Integration #

  1. In ClickHouse, view initial rows of table1:
SELECT * FROM table1
SELECT *
FROM table1

Query id: aee16341-a96f-485b-9aec-689ad5389a34

┌─id─┬─column1─┐
│  1 │ abc     │
│  2 │ def     │
└────┴─────────┘

2 rows in set. Elapsed: 0.011 sec.
  1. Back in MySQL, add a couple of rows to the table1:
INSERT INTO table1
  (id, column1)
VALUES
  (3, 'ghi'),
  (4, 'jkl');
  1. Those two new rows should appear in your ClickHouse table1:
SELECT *
FROM table1

Query id: 75c8bfa7-756c-42ee-8c4a-39a58fc49e11

┌─id─┬─column1─┐
│  1 │ abc     │
│  2 │ def     │
│  3 │ ghi     │
│  4 │ jkl     │
└────┴─────────┘

4 rows in set. Elapsed: 0.007 sec.
  1. Check rows of table2
SELECT *
FROM table2

Query id: 86947b63-90d8-4929-ae24-cd5a84d2a155

┌─txt─┬─column2─┐
│ abc │ 1       │
│ def │ 2       │
└─────┴─────────┘

2 rows in set. Elapsed: 0.005 sec.
  1. Check rows of view1
SELECT *
FROM view1

Query id: 97be5fcb-568e-49ef-b5b0-6f024b650832

┌─id─┬─column1─┬─txt─┬─column2─┐
│  1 │ abc     │ abc │ 1       │
│  2 │ def     │ def │ 2       │
│  3 │ ghi     │     │         │
│  4 │ jkl     │     │         │
└────┴─────────┴─────┴─────────┘

4 rows in set. Elapsed: 0.010 sec.
  1. Back in MySQL, add a couple of rows to table2
INSERT INTO table2
  (txt, column2)
VALUES
  ('ghi', '3'),
  ('jkl', '4'); 
  1. These two new rows should appear in ClickHouse table2:
SELECT *
FROM table2

Query id: 31386b9e-2c6a-4140-97c2-4cd504a4711f

┌─txt─┬─column2─┐
│ abc │ 1       │
│ def │ 2       │
│ ghi │ 3       │
│ jkl │ 4       │
└─────┴─────────┘

4 rows in set. Elapsed: 0.008 sec.
  1. These two new rows also appear in ClickHouse view1:
SELECT *
FROM view1

Query id: 15f03948-1fa7-499c-a345-2f8f47eeae30

┌─id─┬─column1─┬─txt─┬─column2─┐
│  1 │ abc     │ abc │ 1       │
│  2 │ def     │ def │ 2       │
│  3 │ ghi     │ ghi │ 3       │
│  4 │ jkl     │ jkl │ 4       │
└────┴─────────┴─────┴─────────┘

4 rows in set. Elapsed: 0.009 sec.

Setup clickhouse jdbc bridge #

  • To make SQL-server & Oracle server works, please use this customized fork of clickhouse-jdbc-bridge.

  • Download SQL-server & Oracle drivers in the dirver dir.

  • Use the following docker-compse to spin up the DBMSs more easily

  • Assume the dir structure is

    ├── ch-server
    │   └── config
    │       └── jdbc_bridge_config.xml
    ├── docker-compose.yml
    ├── jdbc-bridge
    │   ├── drivers
    │   │   ├── clickhouse-jdbc-0.4.4-all.jar
    │   │   ├── mssql-jdbc-12.2.0.jre8.jar
    │   │   └── ojdbc8-23.2.0.0.jar
    
  • A sample docker-compose file

    version: '2'
    services:
      orclxe:  # su oracle && sqlplus sys/oracle@localhost:1521/XEPDB1 as sysdba
        image: container-registry.oracle.com/database/express:18.4.0-xe
        ports:
          - 1521:1521
        container_name: orclxe
        environment:
          ORACLE_PWD: oracle
        volumes:
          - ${HOME}/data/orclxe:/opt/oracle/oradata  # permission chown -R 54321:54321 ${HOME}/data/orclxe
    
      postgres:
        image: postgres:10.3-alpine
        restart: on-failure
        hostname: postgres
        container_name: postgres
        environment:
          POSTGRES_PASSWORD: password
        ports:
          - "5432:5432"
        volumes:
          - ${HOME}/data/postgres:/var/lib/postgresql/data
    
      mssql:
        image: mcr.microsoft.com/mssql/server:2019-CU13-ubuntu-20.04
        hostname: mssql
        container_name: mssql
        restart: "no"
        environment:
          - ACCEPT_EULA=Y
          - SA_PASSWORD=Tes9ting
        ports:
          - "1433:1433"
        user: "0:0"
        volumes:
          - ${HOME}/data/mssql:/var/opt/mssql  # chown 10001:0 [the-path-of-the-mounted-volume]
    
      mysql:
        image: mysql:5
        hostname: mysql
        container_name: mysql
        restart: "no"
        command: --lower_case_table_names=1
        environment:
          - MYSQL_ROOT_PASSWORD=Tes9ting
          - MYSQL_ROOT_HOST=%
        volumes:
          - ${HOME}/data/mysql:/var/lib/mysql
        ports:
          - "3306:3306"
    
      clickhouse:
        image: clickhouse/clickhouse-server:23.2.3.17
        container_name: clickhouse
        ports:
          - "9000:9000"
          - "8123:8123"
          - "9009:9009"
        environment:
          - CLICKHOUSE_DB=my_database 
          - CLICKHOUSE_USER=clickhouse
          - CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 
          - CLICKHOUSE_PASSWORD=password
        volumes:
          - ${HOME}/data/clickhouse/data:/var/lib/clickhouse
          - ${HOME}/data/clickhouse/log:/val/log/clickhouse-server
          - ${PWD}/ch-server/config:/etc/clickhouse-server/config.d
    
      ch-jdbc-bridge:
        image: clickhouse/jdbc-bridge:2.1
        container_name: ch-jdbc-bridge
        # In general you don't need to define any environment variable
        # Below are all default settings just for demonstration
        environment:
          CONFIG_DIR: config # configuration directory
          SERIAL_MODE: "false" # whether run query in serial mode or not
          HTTPD_CONFIG_FILE: httpd.json # httpd configuration file
          SERVER_CONFIG_FILE: server.json # server configuration file
          VERTX_CONFIG_FILE: vertx.json # vertx configuration file
          DATASOURCE_CONFIG_DIR: datasources # named datasource directory
          DRIVER_DIR: drivers # driver directory
          EXTENSION_DIR: extensions # extension directory
          QUERY_CONFIG_DIR: queries # named query directory
          CUSTOM_DRIVER_LOADER: "true" # whether use custom driver loader or not
          JDBC_BRIDGE_JVM_OPTS: # use CPU and memory allocated by container
        # You may want to keep datasources, queries, SQL scripts, and maybe drivers in a git repo
        volumes:
          - ${PWD}/jdbc-bridge/scripts:/app/scripts
          - ${PWD}/jdbc-bridge/drivers:/app/drivers
        mem_limit: 256m
        restart: "no"
    

Connecting Clickhouse to SQL Server via JDBC #

1. Setting up SQL Server #

  1. Login to sqlserver
/opt/mssql-tools/bin/sqlcmd -U sa
  1. Create a new database in sqlserver:
CREATE DATABASE db_in_sqlserver;
GO

USE db_in_sqlserver;
GO

SELECT DB_NAME();
GO
  1. Create two new table:
CREATE TABLE table1 (id integer primary key, column1 varchar(10));
GO

CREATE TABLE table2 (txt varchar(10) primary key, column2 varchar(10));
GO

CREATE VIEW view1 AS SELECT * from table1 LEFT JOIN table2 ON table1.column1 = table2.txt;
GO

SELECT name, type_desc FROM sys.objects WHERE type_desc IN ('USER_TABLE', 'VIEW');
GO
  1. Add a few rows for testing
INSERT INTO table1(id, column1) VALUES (1, 'abc'), (2, 'def');
GO

INSERT INTO table2(txt, column2)VALUES('abc', '1'), ('def', '2');
GO

2. Define a Table in ClickHouse #

  1. Login to the clickhouse console
clickhouse-client --user default --password ClickHouse123!
  1. Create a new database
CREATE DATABASE sqlserver_in_ch;
use sqlserver_in_ch;
  1. Create a table that use the MySQL
CREATE TABLE table1
(
    id UInt64,
    column1 String
)
ENGINE = JDBC('jdbc:sqlserver://localhost:1433;encrypt=false;databaseName=db_in_sqlserver;user=sa;password=Tes9ting', 'dbo', 'table1');

CREATE TABLE table2
(
    txt String,
    column2 String
)
ENGINE = JDBC('jdbc:sqlserver://localhost:1433;encrypt=false;databaseName=db_in_sqlserver;user=sa;password=Tes9ting', 'dbo', 'table2');

CREATE TABLE view1
(
    id UInt64,
    column1 String,
    txt String,
    column2 String
)
ENGINE = JDBC('jdbc:sqlserver://localhost:1433;encrypt=false;databaseName=db_in_sqlserver;user=sa;password=Tes9ting', 'dbo', 'view1');

3 Test the Integration #

  1. In ClickHouse, view initial rows of table1:
SELECT * FROM table1
SELECT *
FROM table1

Query id: bf00fb71-864c-4e14-9529-2050650db3d2

┌─id─┬─column1─┐
│  1 │ abc     │
│  2 │ def     │
└────┴─────────┘

2 rows in set. Elapsed: 0.053 sec.
  1. Back in SQL-Server, add a couple of rows to the table1:
INSERT INTO table1 (id, column1)VALUES (3, 'ghi'), (4, 'jkl');
GO
  1. Those two new rows should appear in your ClickHouse table1:
SELECT *
FROM table1

Query id: f5bbc5d0-2e2d-4c5c-b1b1-b50ec337c4ae

┌─id─┬─column1─┐
│  1 │ abc     │
│  2 │ def     │
│  3 │ ghi     │
│  4 │ jkl     │
└────┴─────────┘

4 rows in set. Elapsed: 0.067 sec.
  1. Check rows of table2
SELECT *
FROM table2

Query id: 13970fa3-147a-422e-91c1-8f5e4121fcb3

┌─txt─┬─column2─┐
│ abc │ 1       │
│ def │ 2       │
└─────┴─────────┘

2 rows in set. Elapsed: 0.053 sec.
  1. Check rows of view1
SELECT *
FROM view1

Query id: 216f087c-f104-491e-88b0-ba325335701b

┌─id─┬─column1─┬─txt─┬─column2─┐
│  1 │ abc     │ abc │ 1       │
│  2 │ def     │ def │ 2       │
│  3 │ ghi     │     │         │
│  4 │ jkl     │     │         │
└────┴─────────┴─────┴─────────┘

4 rows in set. Elapsed: 1.894 sec.
  1. Back in SQL-Server, add a couple of rows to table2
INSERT INTO table2(txt, column2)VALUES ('ghi', '3'), ('jkl', '4'); 
GO
  1. These two new rows should appear in ClickHouse table2:
SELECT *
FROM table2

Query id: d360193d-7c5a-469d-b381-dbecf7c6e81f

┌─txt─┬─column2─┐
│ abc │ 1       │
│ def │ 2       │
│ ghi │ 3       │
│ jkl │ 4       │
└─────┴─────────┘

4 rows in set. Elapsed: 0.044 sec.
  1. These two new rows also appear in ClickHouse view1:
SELECT *
FROM view1

Query id: fc9375a0-1066-4b17-9461-8ed1ba080c32

┌─id─┬─column1─┬─txt─┬─column2─┐
│  1 │ abc     │ abc │ 1       │
│  2 │ def     │ def │ 2       │
│  3 │ ghi     │ ghi │ 3       │
│  4 │ jkl     │ jkl │ 4       │
└────┴─────────┴─────┴─────────┘

4 rows in set. Elapsed: 2.950 sec.

Connecting Clickhouse to Oracle via JDBC #

Oracle treat SQL identifiers as their upper case form, check here for more details.

Here we don’t use quote, we need to use the upper case as the identifiers(tables, columns) when creating tables in clickhouse.

1. Setting up Oracle Server #

su oracle
sqlplus sys/oracle as sysdba

alter session set container=XEPDB1;
create user db_in_oracle identified by db_in_oracle;
grant dba to db_in_oracle;
grant create session to db_in_oracle;
grant connect, resource to db_in_oracle;
grant all privileges to db_in_oracle;

sqlplus db_in_oracle/db_in_oracle@localhost:1521/XEPDB1
  1. Create two new table:
CREATE TABLE table1 (id integer primary key, column1 varchar(10));

CREATE TABLE table2 (txt varchar(10) primary key, column2 varchar(10));

CREATE VIEW view1 AS SELECT * from table1 LEFT JOIN table2 ON table1.column1 = table2.txt;

COMMIT;
  1. Add a few rows for testing
INSERT INTO table1(id, column1) VALUES (1, 'abc');
INSERT INTO table1(id, column1) VALUES (2, 'def');

INSERT INTO table2(txt, column2)VALUES('abc', '1');
INSERT INTO table2(txt, column2)VALUES('def', '2');

COMMIT;

2. Define a Table in ClickHouse #

  1. Login to the clickhouse console
clickhouse-client --user default --password ClickHouse123!
  1. Create a new database
CREATE DATABASE oracle_in_ch;
use oracle_in_ch;
  1. Create a table that use the Oracle
CREATE TABLE TABLE1
(
    ID UInt64,
    COLUMN1 String
)
ENGINE = JDBC('jdbc:oracle:thin:db_in_oracle/db_in_oracle@localhost:1521/XEPDB1', 'db_in_oracle', 'TABLE1');

CREATE TABLE TABLE2
(
    TXT String,
    COLUMN2 String
)
ENGINE = JDBC('jdbc:oracle:thin:db_in_oracle/db_in_oracle@localhost:1521/XEPDB1', 'db_in_oracle', 'TABLE2');

CREATE TABLE VIEW1
(
    ID UInt64,
    COLUMN1 String,
    TXT String,
    COLUMN2 String
)
ENGINE = JDBC('jdbc:oracle:thin:db_in_oracle/db_in_oracle@localhost:1521/XEPDB1', 'db_in_oracle', 'VIEW1');

3 Test the Integration #

  1. In ClickHouse, view initial rows of table1:
SELECT * FROM TABLE1
SELECT *
FROM TABLE1

Query id: 27312061-1b44-466f-935e-322fca464517

┌─ID─┬─COLUMN1─┐
│  1 │ abc     │
│  2 │ def     │
└────┴─────────┘

2 rows in set. Elapsed: 0.140 sec.
  1. Back in Oracle server, add a couple of rows to the TABLE1:
INSERT INTO table1(id, column1) VALUES (3, 'ghi');
INSERT INTO table1(id, column1) VALUES (4, 'jkl');
COMMIT;
  1. Those two new rows should appear in your ClickHouse TABLE1:
SELECT *
FROM TABLE1

Query id: f2541173-90e3-4be8-a8ed-34159c41c891

┌─ID─┬─COLUMN1─┐
│  1 │ abc     │
│  2 │ def     │
│  3 │ ghi     │
│  4 │ jkl     │
└────┴─────────┘

4 rows in set. Elapsed: 0.084 sec.
  1. Check rows of TABLE2
SELECT *
FROM TABLE2

Query id: 85eed8dc-dfb1-4f6d-926e-e86979cabcd0

┌─TXT─┬─COLUMN2─┐
│ abc │ 1       │
│ def │ 2       │
└─────┴─────────┘

2 rows in set. Elapsed: 0.062 sec.
  1. Check rows of VIEW1
SELECT *
FROM VIEW1

Query id: b9102b77-7048-4cdb-9491-cc1086586e29

┌─ID─┬─COLUMN1─┬─TXT─┬─COLUMN2─┐
│  1 │ abc     │ abc │ 1       │
│  2 │ def     │ def │ 2       │
│  3 │ ghi     │     │         │
│  4 │ jkl     │     │         │
└────┴─────────┴─────┴─────────┘

4 rows in set. Elapsed: 0.092 sec.
  1. Back in Oracle Server, add a couple of rows to TABLE2
INSERT INTO table2(txt, column2)VALUES ('ghi', '3');
INSERT INTO table2(txt, column2)VALUES ('jkl', '4'); 
COMMIT
  1. These two new rows should appear in ClickHouse TABLE2:
SELECT *
FROM TABLE2

Query id: c5eeb415-453c-4981-b635-7098677cd22a

┌─TXT─┬─COLUMN2─┐
│ abc │ 1       │
│ def │ 2       │
│ ghi │ 3       │
│ jkl │ 4       │
└─────┴─────────┘

4 rows in set. Elapsed: 0.075 sec.
  1. These two new rows also appear in ClickHouse VIEW1:
SELECT *
FROM VIEW1

Query id: d900a9c8-f526-4aa2-a190-6623210312b4

┌─ID─┬─COLUMN1─┬─TXT─┬─COLUMN2─┐
│  1 │ abc     │ abc │ 1       │
│  2 │ def     │ def │ 2       │
│  3 │ ghi     │ ghi │ 3       │
│  4 │ jkl     │ jkl │ 4       │
└────┴─────────┴─────┴─────────┘

4 rows in set. Elapsed: 0.056 sec.

Connecting Clickhouse to MongoDB #

1. Setting up MongoDB #

  1. Login to mongo
mongo -uroot -p
  1. Create two new collections with some data under db_in_mogno database:
use db_in_mongo

db.collection1.insertMany([{id: 1, column1: "abc"}, {id: 2, column2: "def"}]);
db.collection2.insertMany([{txt: "abc", column2: "1"}, {txt: "def", column2: "2"}]);

show dbs
show collections

2. Define a Table in ClickHouse #

  1. Login to the clickhouse console
clickhouse-client --user default --password ClickHouse123!
  1. Create a new database
CREATE DATABASE mongo_in_ch;
use mongo_in_ch;
  1. Create a table that use the Mongo
CREATE TABLE table1
(
    id UInt64,
    column1 String
)
ENGINE = MongoDB('mongodb:27017', 'db_in_mongo', 'collection1', 'root', 'Tes9ting', 'authSource=admin');

CREATE TABLE table2
(
    txt String,
    column2 String
)
ENGINE = MongoDB('mongodb:27017', 'db_in_mongo', 'collection2', 'root', 'Tes9ting', 'authSource=admin');

3 Test the Integration #

  1. In ClickHouse, view initial rows of table1:
SELECT * FROM table1
SELECT *
FROM table1

Query id: ef4be0ad-05d0-4bb9-ad4c-4d33c27ee044

┌─id─┬─column1─┐
│  1 │ abc     │
│  2 │         │
└────┴─────────┘

2 rows in set. Elapsed: 0.051 sec.
  1. Back in Mongo, add a couple of rows to the collection1:
db.collection1.insertMany([{id: 3, column1: "ghi"}, {id: 4, column1: "jkl"}]);
  1. Those two new rows should appear in your ClickHouse table1:
SELECT *
FROM table1

Query id: e7368646-af24-437d-952c-9227ce9dec66

┌─id─┬─column1─┐
│  1 │ abc     │
│  2 │         │
│  3 │ ghi     │
│  4 │ jkl     │
└────┴─────────┘

4 rows in set. Elapsed: 0.005 sec.
  1. Check rows of table2
SELECT *
FROM table2

Query id: 70b9b73f-1bef-4883-beaa-202e588ce5ab

┌─txt─┬─column2─┐
│ abc │ 1       │
│ def │ 2       │
└─────┴─────────┘

2 rows in set. Elapsed: 0.004 sec.