Smart Kettle v1.0

Smart Kettle - Kettle big data scheduling service monitoring platform based on Springboot

Downcodes editor

Smart Kettle is a Kettle big data scheduling service monitoring platform based on the currently popular Springboot microservice back-end architecture on the Internet and using rich front-end Vue, iView and other component technologies. It is designed to solve the pain point of being unable to configure, schedule, and monitor Kettle through the Web in enterprise data extraction business scenarios.

Advantages of Smart Kettle

Smart Kettle has packaged and optimized the use of Kettle for common pain points in enterprise data processing business, so that it can also have basic Kettle job and conversion configuration, scheduling, and monitoring functions on the Web side. This can, to a large extent, assist enterprises in completing the ETL (extraction, transformation, and processing) capabilities of data in different business scenarios.

Note: This system is not a re-engineering of Kettle source code, but uses Kettle's API to achieve a gorgeous transformation of Kettle's Web-side functions.

Features

1. Lightweight Kettle web-side scheduling and monitoring platform

Supports custom template settings for jobs and transitions.

Supports multi-tasking template copying of jobs and transformations.

Supports synchronization of GUI side configurations of jobs and conversions to the Web side.

Supports custom path configuration of logs for jobs and conversions.

Supports log file download management for jobs and conversions.

Support file library resource library configuration.

Support database resource library configuration.

Supports remote resource library configuration of SFTP/FTP/FTPS.

Supports resource library directory tree viewing.

Supports cluster scheduling of jobs and transformations (remote subserver calls).

Supports scheduling configuration of jobs and transformations.

Supports monitoring and management of jobs and conversions.

Supports local execution of jobs and transformations.

Supports remote execution of jobs and transformations.

Supports scheduled configuration of jobs and conversions.

Supports real-time monitoring of jobs and conversions.

Support Kettle's web resource library management.

Supports custom thread pool settings and parallel processing of tasks.

Supports alarm monitoring and log management of Kettle tasks.

Provides rich custom settings for business library and dictionary library.

Provides complete, real-time dispatch monitoring.

Provide system users, roles, and permissions management.

Provides real-time monitoring capabilities for Druid database query scripts.

Supports Kettle 7.0.1+ and above, the current version is 9.2.0.0-179 (note: kettle-password-encoder-plugins need to be configured, this system has been integrated, no further configuration is required).

2. Front-end and back-end separation design

The backend architecture is implemented based on Springboot.

The server can publish REST services.

The front-end architecture is based on VUE, and the data components are richer and easier to maintain.

The client obtains server data through AJAX and renders the interface.

3. Implementation based on microservice technology

The back-end architecture uses the self-built x-common-base framework.

The backend encapsulates x-kettle-core, a powerful interface plug-in based on Kettle.

The backend framework is implemented based on Springboot + Mybatis.

The back-end framework is easy to migrate, secondary develop, and maintain.

4. Front-end implementation based on progressive VUE framework

Functional components are easy to develop and maintain.

It is easy for novices to get started and get started in a short time.

The interface is more beautiful and the components are richer.

Environmental requirements

Maven3+

JDK1.8+

MySQL5.7+

Summarize

Smart Kettle provides a convenient and efficient Kettle big data scheduling service monitoring platform to help enterprises manage and monitor data processing processes more efficiently. Its front-end and back-end separation design, based on the implementation of microservices and VUE framework, provides enterprises with flexible and scalable solutions.