填坑唯品会分布式调度Saturn

算法之名

发布于 2019-08-20 16:07:17

1.2K0

发布于 2019-08-20 16:07:17

文章被收录于专栏：算法之名算法之名

这里面说要安装node.js 8.7.0+跟npm 5.4.2+

但其实node.js 8.7.0就已经包含了npm 5.4.2,首先下载node.js 8.7.0

wget https://nodejs.org/dist/v8.7.0/node-v8.7.0-linux-x64.tar.gz

解压tar -xzvf node-v8.7.0-linux-x64.tar.gz

mv node-v8.7.0-linux-x64 /usr/local/node-8.7

cd /usr/local/node-8.7/

建立软链接

ln -s /usr/local/node-8.7/lib/node_modules/npm/bin/npm-cli.js /usr/local/bin/npm

ln -s /usr/local/node-8.7/bin/node /usr/local/bin/node

修改vim /etc/profile

PATH=/usr/local/node-8.7/bin:$PATH

export PATH

保存退出，source /etc/profile

[root@host2 node-8.7]# node -v v8.7.0 [root@host2 node-8.7]# npm -v 5.4.2

这样我们可以看到node跟npm就装好了。

因为我是从docker启动的，而这里需要有一个docker-compose，这里我们需要来安装docker-compose.

要安装docker-compose需要先安装pip.

yum -y install epel-release

yum -y install python-pip

升级pip

pip install --upgrade pip

查看pip版本号

[root@host2 node-8.7]# pip -V （这里是大写的V） pip 18.1 from /usr/lib/python2.7/site-packages/pip (python 2.7)

这个时候用pip来安装docker-compose是肯定会报错的

pip install docker-compose

ipapython 4.5.0 requires pyldap>=2.4.15, which is not installed. ipapython 4.5.0 has requirement dnspython>=1.15, but you'll have dnspython 1.12.0 which is incompatible.

Cannot uninstall 'requests'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

根据提示我们来安装pyldap也一样会报错

pip install pyldap

Cannot uninstall 'requests'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

ipapython 4.5.0 has requirement dnspython>=1.15, but you'll have dnspython 1.12.0 which is incompatible.

Cannot uninstall 'python-ldap'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

根据提示dnspython版本不对，要进行升级

pip install --ignore-installed dnspython

安装yum install python-devel

安装yum install openldap-devel

升级pip install --ignore-installed python-ldap

安装pip install pyldap

现在可以安装docker-compose了

pip install docker-compose --ignore-installed requests

[root@host2 node-8.7]# docker-compose -version docker-compose version 1.23.1, build b02f130

修改saturn-docker目录下的quickstart-docker.sh

vim quickstart-docker.sh

把docker-compose up改成从后台启动

nohup docker-compose up &

保存退出。

当然我们还要安装docker（如果你没装的话）

yum install docker

修改quickstart-docker.sh为可执行文件

chmod 755 quickstart-docker.sh

运行该文件

./quickstart-docker.sh

我们在浏览器就可以看到管理界面了（端口号9088）

查看docker进程可以看到他启了什么

[root@host1 saturn-docker]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d200d64a2b02 saturn/demo-init "/saturn-executor/..." 27 hours ago Up 25 hours demo-init 0a92685fb26c saturn/saturn-console "java -DSATURN_CON..." 27 hours ago Up 25 hours 0.0.0.0:2181->2181/tcp, 0.0.0.0:9088->9088/tcp saturn_console 08f66a0247e9 saturn/saturn-db "docker-entrypoint..." 27 hours ago Up 25 hours 33060/tcp, 0.0.0.0:3308->3306/tcp saturn_db

Java开发Job填坑

新建一个maven项目（非springboot项目）

pom

<dependencies>
    <dependency>
        <groupId>com.vip.saturn</groupId>
        <artifactId>saturn-job-api</artifactId>
        <!-- 修改成指定版本 -->
        <version>3.1.0</version>
    </dependency>
</dependencies>
<build>
    <plugins>
        <plugin>
            <groupId>com.vip.saturn</groupId>
            <artifactId>saturn-plugin</artifactId>
            <!-- 版本与saturn-job-api一致 -->
            <version>3.1.0</version>
        </plugin>
    </plugins>
</build>

java类

package com.guanjian;


import com.vip.saturn.job.AbstractSaturnJavaJob;
import com.vip.saturn.job.SaturnJobExecutionContext;
import com.vip.saturn.job.SaturnJobReturn;

/**
 * Created by Administrator on 2018/11/5.
 */
public class DemoJob extends AbstractSaturnJavaJob {
    @Override
    public SaturnJobReturn handleJavaJob(String s, Integer integer, String s1, SaturnJobExecutionContext saturnJobExecutionContext) throws InterruptedException {
        return new SaturnJobReturn("我是分片" + integer + "处理结果");
    }
}

Windows开发环境下修改C:\Windows\System32\drivers\etc\hosts

添加192.168.5.129 console的映射（很重要，IP地址为你安装的服务器，教程上没有说）

在linux系统下同样要在/etc/hosts下添加该映射

然后刷新网卡service network restart

添加maven参数

saturn:run -Dnamespace=saturn-it.vip.com -DexecutorName=exec01 -DVIP_SATURN_CONSOLE_URI=http://192.168.5.129:9088

依据教程我就不多说了，可以运行起来

打包：（在Windows CMD命令行下）

E:\saturntest>mvn saturn:zip [INFO] Scanning for projects... [INFO] [INFO] ----------------------< com.guanjian:saturntest >----------------------- [INFO] Building saturntest 1.0-SNAPSHOT [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] >>> saturn-plugin:3.1.0:zip (default-cli) > package @ saturntest >>> Downloading from central: https://repo.maven.apache.org/maven2/com/vip/saturn/saturn-job-api/3.1.0/saturn-job-api-3.1.0.pom Downloaded from central: https://repo.maven.apache.org/maven2/com/vip/saturn/saturn-job-api/3.1.0/saturn-job-api-3.1.0.pom (463 B at 241 B/s) Downloading from central: https://repo.maven.apache.org/maven2/com/vip/saturn/saturn-job-api/3.1.0/saturn-job-api-3.1.0.jar Downloaded from central: https://repo.maven.apache.org/maven2/com/vip/saturn/saturn-job-api/3.1.0/saturn-job-api-3.1.0.jar (16 kB at 16 kB/s) [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ saturntest --- [WARNING] Using platform encoding (GBK actually) to copy filtered resources, i.e. build is platform dependent! [INFO] Copying 0 resource [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ saturntest --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ saturntest --- [WARNING] Using platform encoding (GBK actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory E:\saturntest\src\test\resources [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ saturntest --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ saturntest --- [INFO] No tests to run. [INFO] [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ saturntest --- [INFO] Building jar: E:\saturntest\target\saturntest-1.0-SNAPSHOT.jar [INFO] [INFO] <<< saturn-plugin:3.1.0:zip (default-cli) < package @ saturntest <<< [INFO] [INFO] [INFO] --- saturn-plugin:3.1.0:zip (default-cli) @ saturntest ---

[INFO] Packing the saturn job into a zip file: version:3.1.0 [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 43.362 s [INFO] Finished at: 2018-11-06T15:41:16+08:00 [INFO] ------------------------------------------------------------------------

此时我们可以看到