未读消息消息

购物车

我的订单

个人中心

店铺

我的订单收藏

拍卖

拍卖交易我的竞拍收藏

我的好友资金账户

卖家中心

客服 |

帮助中心 9:00-20:30 在线留言

客服电话

010-89648155

服务时间

客服咨询 8:00-21:00

纠纷处理 9:00-21:00

图书审核 9:00-18:00

监督与建议

请选择

手机孔网

精通Apache Pulsar：可伸缩云原生事件流实践

作者: [美] Jowanza Joseph(乔万扎·约瑟夫

出版社: 电子工业出版社

出版时间: 2023-12

版次: 1

ISBN: 9787121467172

定价: 89.00

装帧: 其他

开本: 16开

纸张: 胶版纸

页数: 224页

分类: 计算机与互联网

本书专为想要学习并掌握Apache Pulsar的读者打造，是一本入门图书。全书共有13章内容。第1章和第2章，从生活实例中引入实时消息系统和事件驱动的概念，帮助大家理解Apache Pulsar系统存在的必要性，以及Pulsar的独特性和价值定位。第3章介绍了Pulsar在雅虎的起源故事、设计理念与应用场景。第4章探究了Pulsar内部的主要组件及其工作原理，帮助大家构建对Pulsar的整体架构认知。第5章到第10章，详细介绍了Pulsar消费者、生产者的工作原理，应用于生产和消费消息且与第三方系统交互的框架Pulsar IO，以及Pulsar Functions、分层存储、Pulsar SQL等Pulsar的独特理念和关键特性。第11章与第12章，展示了如何在诸多环境中部署Pulsar，以及在运维Pulsar的过程中需要关注的指标。在第13章中，作者结合个人经验阐述了Pulsar未来值得探索的领域和方向。本书配图生动形象，实例丰富，不仅提供了很多代码示例，还提供了多篇附录，以作参考。 Jowanza Joseph 是一位就职于 Finicity 的软件工程师，他负责领导 Finicity 开放银行平台的网格开发。Jowanza 曾在多个项目中使用 Apache Pulsar，并利用基于 Pulsar 构建的全托管消息和流处理平台实现了日均处理十亿量级的消息。近十年中， Jowanza 在 Pluralsight 等公司的工作中都使用过Apache Kafka、Akka、Kubernetes 及其他流技术与消息技术。他还曾在 Strange Loop、Abstractions、开源峰会及 O’Reilly 的 Strata Data & AI 大会上做过技术分享。蔡正昕，曾在亚马逊工作， Pulsar 活跃贡献者。后来加入了 StreamNative，成为 Cloud 工程师。

蔡正昕，曾在亚马逊工作， Pulsar 活跃贡献者。后来加入了 StreamNative，成为 Cloud 工程师。

王殿进，目前在 StreamNative 负责 Apache Pulsar 社区与市场工作。曾在云计算厂商、SaaS 公司从事过开发者社区运营与用户增长工作。

徐文涛，曾任Talkdesk 知识管理工程师，为多个企业级 SaaS、CCaaS 与 PaaS 产品撰写并审核英文文档。曾任某世界五百强本地化团队终审负责人，原 KubeSphere 开源容器平台本地化、文档与内容策划负责人。曾获美国 DZone 专栏银牌作家，人事部 CATTI 二级口译、二级笔译，联合国 UNLPP 二级笔译，Linux 基金会认证 Kubernetes 管理员 (CKA)、应用开发者 (CKAD) 与安全专家 (CKS)等行业资格或荣誉。

前言 ....................................................................................................xvii

第 1 章　实时消息的价值 ........................................................................ 1

流动的数据 ............................................................................................................1

资源效率 ................................................................................................................3

使用场景 ................................................................................................................4

银行 .................................................................................................................5

医疗 .................................................................................................................6

安全 .................................................................................................................7

物联网 .............................................................................................................8

小结 .....................................................................................................................10

第 2 章　事件流和事件代理 ................................................................... 11

发布 / 订阅 ........................................................................................................... 11

队列 .....................................................................................................................13

故障类型 ..............................................................................................................15

推送与轮询 ..........................................................................................................15

为什么需要 Pulsar ................................................................................................16

流和队列的统一 ............................................................................................16

模块化 ...........................................................................................................16

性能 ...............................................................................................................18

小结 .....................................................................................................................19

第 3 章　认识 Pulsar ............................................................................ 20

Pulsar 的起源 .......................................................................................................20

Pulsar 的设计理念 ................................................................................................21

多租户 ...........................................................................................................22

跨地域复制 ...................................................................................................23

性能 ...............................................................................................................27

模块化 ...........................................................................................................27

Pulsar 生态 ...........................................................................................................28

Pulsar Functions ............................................................................................28

Pulsar IO .......................................................................................................29

Pulsar SQL ....................................................................................................30

Pulsar 成功案例 ...................................................................................................30

雅虎日本 .......................................................................................................30

Splunk ...........................................................................................................31

Iterable ..........................................................................................................32

小结 .....................................................................................................................33

第 4 章　Pulsar 内部组件 ..................................................................... 34

Broker ..................................................................................................................34

消息缓存 .......................................................................................................35

与 BookKeeper、ZooKeeper 交互 .................................................................37

Schema 校验 ..................................................................................................38

Broker 间的通信 ............................................................................................38

Pulsar Functions 和 Pulsar IO ........................................................................39

Apache BookKeeper .............................................................................................39

预写式日志 ...................................................................................................45

消息存储 .......................................................................................................45

对象 /Blob 存储 .............................................................................................46

Pravega ..........................................................................................................47

Majordodo .....................................................................................................47

Apache ZooKeeper ...............................................................................................48

命名服务 .......................................................................................................49

配置管理 .......................................................................................................49

选主 ...............................................................................................................50

通知系统 .......................................................................................................50

Apache Kafka ................................................................................................50

Apache Druid .................................................................................................51

Pulsar Proxy .........................................................................................................52

Java 虚拟机（JVM） .............................................................................................53

Netty ..............................................................................................................54

Apache Spark .................................................................................................55

Apache Lucene ..............................................................................................55

小结 .....................................................................................................................55

第 5 章　消费者 .................................................................................... 56

成为消费者意味着什么 ........................................................................................56

订阅 .....................................................................................................................57

独占 ...............................................................................................................60

共享 ...............................................................................................................61

键共享 ...........................................................................................................62

故障转移 .......................................................................................................65

确认 .....................................................................................................................66

单条确认 .......................................................................................................66

累积确认 .......................................................................................................67

Schema .................................................................................................................67

消费者 Schema 管理 .....................................................................................68

消费模式 ..............................................................................................................68

批处理 ...........................................................................................................68

分块 ...............................................................................................................69

高级配置 ..............................................................................................................70

延迟消息 .......................................................................................................70

保留策略 .......................................................................................................70

堆积限额 .......................................................................................................71

配置消费者 ..........................................................................................................72

回放 ...............................................................................................................73

死信主题 .......................................................................................................75

重试主题 .......................................................................................................76

小结 .....................................................................................................................76

第 6 章　生产者 .................................................................................... 77

同步生产者 ..........................................................................................................77

异步生产者 ..........................................................................................................78

生产者路由 ..........................................................................................................79

轮询路由 .......................................................................................................79

单分区路由 ...................................................................................................80

定制分区路由 ................................................................................................81

生产者配置 ..........................................................................................................83

topicName .....................................................................................................83

producerName ...............................................................................................83

sendTimeoutMs .............................................................................................83

blockIfQueueFull ...........................................................................................84

maxPendingMessages ....................................................................................84

maxPendingMessagesAcrossPartitions ...........................................................84

messageRoutingMode ....................................................................................84

hashingScheme ..............................................................................................84

cryptoFailureAction .......................................................................................85

batchingMaxPublishDelayMicros ..................................................................85

batchingMaxMessages ...................................................................................85

batchingEnabled ............................................................................................85

compressionType ...........................................................................................85

Schema on Write ..................................................................................................86

使用 Schema Registry ....................................................................................86

非持久化主题 ......................................................................................................89

使用场景 .......................................................................................................90

使用非持久化主题 ........................................................................................90

事务 .....................................................................................................................91

小结 .....................................................................................................................92

第 7 章　Pulsar IO ............................................................................... 93

Pulsar IO 架构 ......................................................................................................93

运行时 ...........................................................................................................94

性能考量 .......................................................................................................94

使用场景 ..............................................................................................................95

简单事件处理管道 ........................................................................................95

变化数据捕获 ................................................................................................96

考量 .....................................................................................................................97

消息序列化 ...................................................................................................97

管道稳定性 ...................................................................................................97

故障处理 .......................................................................................................98

示例 .....................................................................................................................99

Elasticsearch ..................................................................................................99

Netty ............................................................................................................ 100

创建自己的连接器 ............................................................................................. 101

TimescaleDB ............................................................................................... 101

小结 ................................................................................................................... 102

第 8 章　Pulsar Functions .................................................................. 103

流处理 ................................................................................................................ 103

Pulsar Functions 架构 ......................................................................................... 105

运行时 ......................................................................................................... 105

隔离性 ......................................................................................................... 106

在 Kubernetes 上部署 Pulsar Functions 的隔离性 .............................................. 107

使用场景 ............................................................................................................ 107

创建 Pulsar Functions .................................................................................. 108

简单事件处理 .............................................................................................. 108

主题清理 ..................................................................................................... 110

主题数据统计 .............................................................................................. 113

小结 ................................................................................................................... 115

第 9 章　分层存储 .............................................................................. 116

在云端存储数据 ................................................................................................. 117

对象存储 ..................................................................................................... 118

使用场景 ............................................................................................................ 120

复制 ............................................................................................................. 120

CQRS .......................................................................................................... 121

灾难恢复 ..................................................................................................... 122

卸载数据 ............................................................................................................ 122

Pulsar Offloader ........................................................................................... 122

检索已卸载的数据 ............................................................................................. 129

与对象存储数据交互 ................................................................................... 129

重填主题 ..................................................................................................... 129

使用 Pulsar 客户端 ...................................................................................... 130

小结 ................................................................................................................... 131

第 10 章　Pulsar SQL ........................................................................ 132

流即表 ................................................................................................................ 133

SQL-on-Anything 引擎 ...................................................................................... 135

Apache Flink: 另一种视角 ........................................................................... 136

Presto/Trino ................................................................................................. 137

Pulsar SQL 的工作原理 ...................................................................................... 138

配置 Pulsar SQL ................................................................................................. 140

性能考量 ............................................................................................................ 142

小结 ................................................................................................................... 143

第 11 章　部署 Pulsar ........................................................................ 144

Docker ................................................................................................................ 144

裸金属 ................................................................................................................ 146

最低要求 ..................................................................................................... 146

开始安装 ..................................................................................................... 146

部署 ZooKeeper........................................................................................... 147

启动 BookKeeper ........................................................................................ 148

启动 Pulsar .................................................................................................. 148

公有云厂商 ........................................................................................................ 149

AWS ............................................................................................................ 151

Azure ........................................................................................................... 152

Google Cloud Platform ................................................................................ 153

Kubernetes ......................................................................................................... 153

小结 ................................................................................................................... 155

第 12 章　运维 Pulsar ........................................................................ 156

Apache BookKeeper 指标 .................................................................................. 157

服务器指标 ................................................................................................. 157

Journal 指标 ................................................................................................ 158

存储指标 ..................................................................................................... 159

Apache ZooKeeper 指标 ..................................................................................... 159

服务器指标 ................................................................................................. 160

请求指标 ..................................................................................................... 161

主题指标 ............................................................................................................ 161

消费者指标 ........................................................................................................ 163

Pulsar 事务指标 ................................................................................................. 163

Pulsar Functions 指标 ......................................................................................... 164

高级运维技巧 .................................................................................................... 165

拦截器和追踪机制 ...................................................................................... 165

Pulsar SQL 指标 .......................................................................................... 167

指标转发 ............................................................................................................ 169

监控面板 ..................................................................................................... 169

小结 ................................................................................................................... 170

第 13 章　未来 ................................................................................... 171

编程语言支持 .................................................................................................... 171

扩展接口 ..................................................................................................... 171

增强 Pulsar Functions .................................................................................. 172

架构简化 / 扩展 ........................................................................................... 175

消息平台桥接器 .......................................................................................... 177

小结 ................................................................................................................... 180

附录 A Pulsar Admin API ................................................................. 181

附录 B Pulsar Admin CLI ................................................................. 189

附录 C　跨地域复制 ........................................................................... 192

附录 D Pulsar 中的安全、认证和鉴权 ............................................... 198

关于作者 ............................................................................................ 202

封面介绍 ............................................................................................ 202
内容简介:
本书专为想要学习并掌握Apache Pulsar的读者打造，是一本入门图书。全书共有13章内容。第1章和第2章，从生活实例中引入实时消息系统和事件驱动的概念，帮助大家理解Apache Pulsar系统存在的必要性，以及Pulsar的独特性和价值定位。第3章介绍了Pulsar在雅虎的起源故事、设计理念与应用场景。第4章探究了Pulsar内部的主要组件及其工作原理，帮助大家构建对Pulsar的整体架构认知。第5章到第10章，详细介绍了Pulsar消费者、生产者的工作原理，应用于生产和消费消息且与第三方系统交互的框架Pulsar IO，以及Pulsar Functions、分层存储、Pulsar SQL等Pulsar的独特理念和关键特性。第11章与第12章，展示了如何在诸多环境中部署Pulsar，以及在运维Pulsar的过程中需要关注的指标。在第13章中，作者结合个人经验阐述了Pulsar未来值得探索的领域和方向。本书配图生动形象，实例丰富，不仅提供了很多代码示例，还提供了多篇附录，以作参考。
作者简介:
Jowanza Joseph 是一位就职于 Finicity 的软件工程师，他负责领导 Finicity 开放银行平台的网格开发。Jowanza 曾在多个项目中使用 Apache Pulsar，并利用基于 Pulsar 构建的全托管消息和流处理平台实现了日均处理十亿量级的消息。近十年中， Jowanza 在 Pluralsight 等公司的工作中都使用过Apache Kafka、Akka、Kubernetes 及其他流技术与消息技术。他还曾在 Strange Loop、Abstractions、开源峰会及 O’Reilly 的 Strata Data & AI 大会上做过技术分享。蔡正昕，曾在亚马逊工作， Pulsar 活跃贡献者。后来加入了 StreamNative，成为 Cloud 工程师。

蔡正昕，曾在亚马逊工作， Pulsar 活跃贡献者。后来加入了 StreamNative，成为 Cloud 工程师。

王殿进，目前在 StreamNative 负责 Apache Pulsar 社区与市场工作。曾在云计算厂商、SaaS 公司从事过开发者社区运营与用户增长工作。

徐文涛，曾任Talkdesk 知识管理工程师，为多个企业级 SaaS、CCaaS 与 PaaS 产品撰写并审核英文文档。曾任某世界五百强本地化团队终审负责人，原 KubeSphere 开源容器平台本地化、文档与内容策划负责人。曾获美国 DZone 专栏银牌作家，人事部 CATTI 二级口译、二级笔译，联合国 UNLPP 二级笔译，Linux 基金会认证 Kubernetes 管理员 (CKA)、应用开发者 (CKAD) 与安全专家 (CKS)等行业资格或荣誉。
目录:
前言 ....................................................................................................xvii

第 1 章　实时消息的价值 ........................................................................ 1

流动的数据 ............................................................................................................1

资源效率 ................................................................................................................3

使用场景 ................................................................................................................4

银行 .................................................................................................................5

医疗 .................................................................................................................6

安全 .................................................................................................................7

物联网 .............................................................................................................8

小结 .....................................................................................................................10

第 2 章　事件流和事件代理 ................................................................... 11

发布 / 订阅 ........................................................................................................... 11

队列 .....................................................................................................................13

故障类型 ..............................................................................................................15

推送与轮询 ..........................................................................................................15

为什么需要 Pulsar ................................................................................................16

流和队列的统一 ............................................................................................16

模块化 ...........................................................................................................16

性能 ...............................................................................................................18

小结 .....................................................................................................................19

第 3 章　认识 Pulsar ............................................................................ 20

Pulsar 的起源 .......................................................................................................20

Pulsar 的设计理念 ................................................................................................21

多租户 ...........................................................................................................22

跨地域复制 ...................................................................................................23

性能 ...............................................................................................................27

模块化 ...........................................................................................................27

Pulsar 生态 ...........................................................................................................28

Pulsar Functions ............................................................................................28

Pulsar IO .......................................................................................................29

Pulsar SQL ....................................................................................................30

Pulsar 成功案例 ...................................................................................................30

雅虎日本 .......................................................................................................30

Splunk ...........................................................................................................31

Iterable ..........................................................................................................32

小结 .....................................................................................................................33

第 4 章　Pulsar 内部组件 ..................................................................... 34

Broker ..................................................................................................................34

消息缓存 .......................................................................................................35

与 BookKeeper、ZooKeeper 交互 .................................................................37

Schema 校验 ..................................................................................................38

Broker 间的通信 ............................................................................................38

Pulsar Functions 和 Pulsar IO ........................................................................39

Apache BookKeeper .............................................................................................39

预写式日志 ...................................................................................................45

消息存储 .......................................................................................................45

对象 /Blob 存储 .............................................................................................46

Pravega ..........................................................................................................47

Majordodo .....................................................................................................47

Apache ZooKeeper ...............................................................................................48

命名服务 .......................................................................................................49

配置管理 .......................................................................................................49

选主 ...............................................................................................................50

通知系统 .......................................................................................................50

Apache Kafka ................................................................................................50

Apache Druid .................................................................................................51

Pulsar Proxy .........................................................................................................52

Java 虚拟机（JVM） .............................................................................................53

Netty ..............................................................................................................54

Apache Spark .................................................................................................55

Apache Lucene ..............................................................................................55

小结 .....................................................................................................................55

第 5 章　消费者 .................................................................................... 56

成为消费者意味着什么 ........................................................................................56

订阅 .....................................................................................................................57

独占 ...............................................................................................................60

共享 ...............................................................................................................61

键共享 ...........................................................................................................62

故障转移 .......................................................................................................65

确认 .....................................................................................................................66

单条确认 .......................................................................................................66

累积确认 .......................................................................................................67

Schema .................................................................................................................67

消费者 Schema 管理 .....................................................................................68

消费模式 ..............................................................................................................68

批处理 ...........................................................................................................68

分块 ...............................................................................................................69

高级配置 ..............................................................................................................70

延迟消息 .......................................................................................................70

保留策略 .......................................................................................................70

堆积限额 .......................................................................................................71

配置消费者 ..........................................................................................................72

回放 ...............................................................................................................73

死信主题 .......................................................................................................75

重试主题 .......................................................................................................76

小结 .....................................................................................................................76

第 6 章　生产者 .................................................................................... 77

同步生产者 ..........................................................................................................77

异步生产者 ..........................................................................................................78

生产者路由 ..........................................................................................................79

轮询路由 .......................................................................................................79

单分区路由 ...................................................................................................80

定制分区路由 ................................................................................................81

生产者配置 ..........................................................................................................83

topicName .....................................................................................................83

producerName ...............................................................................................83

sendTimeoutMs .............................................................................................83

blockIfQueueFull ...........................................................................................84

maxPendingMessages ....................................................................................84

maxPendingMessagesAcrossPartitions ...........................................................84

messageRoutingMode ....................................................................................84

hashingScheme ..............................................................................................84

cryptoFailureAction .......................................................................................85

batchingMaxPublishDelayMicros ..................................................................85

batchingMaxMessages ...................................................................................85

batchingEnabled ............................................................................................85

compressionType ...........................................................................................85

Schema on Write ..................................................................................................86

使用 Schema Registry ....................................................................................86

非持久化主题 ......................................................................................................89

使用场景 .......................................................................................................90

使用非持久化主题 ........................................................................................90

事务 .....................................................................................................................91

小结 .....................................................................................................................92

第 7 章　Pulsar IO ............................................................................... 93

Pulsar IO 架构 ......................................................................................................93

运行时 ...........................................................................................................94

性能考量 .......................................................................................................94

使用场景 ..............................................................................................................95

简单事件处理管道 ........................................................................................95

变化数据捕获 ................................................................................................96

考量 .....................................................................................................................97

消息序列化 ...................................................................................................97

管道稳定性 ...................................................................................................97

故障处理 .......................................................................................................98

示例 .....................................................................................................................99

Elasticsearch ..................................................................................................99

Netty ............................................................................................................ 100

创建自己的连接器 ............................................................................................. 101

TimescaleDB ............................................................................................... 101

小结 ................................................................................................................... 102

第 8 章　Pulsar Functions .................................................................. 103

流处理 ................................................................................................................ 103

Pulsar Functions 架构 ......................................................................................... 105

运行时 ......................................................................................................... 105

隔离性 ......................................................................................................... 106

在 Kubernetes 上部署 Pulsar Functions 的隔离性 .............................................. 107

使用场景 ............................................................................................................ 107

创建 Pulsar Functions .................................................................................. 108

简单事件处理 .............................................................................................. 108

主题清理 ..................................................................................................... 110

主题数据统计 .............................................................................................. 113

小结 ................................................................................................................... 115

第 9 章　分层存储 .............................................................................. 116

在云端存储数据 ................................................................................................. 117

对象存储 ..................................................................................................... 118

使用场景 ............................................................................................................ 120

复制 ............................................................................................................. 120

CQRS .......................................................................................................... 121

灾难恢复 ..................................................................................................... 122

卸载数据 ............................................................................................................ 122

Pulsar Offloader ........................................................................................... 122

检索已卸载的数据 ............................................................................................. 129

与对象存储数据交互 ................................................................................... 129

重填主题 ..................................................................................................... 129

使用 Pulsar 客户端 ...................................................................................... 130

小结 ................................................................................................................... 131

第 10 章　Pulsar SQL ........................................................................ 132

流即表 ................................................................................................................ 133

SQL-on-Anything 引擎 ...................................................................................... 135

Apache Flink: 另一种视角 ........................................................................... 136

Presto/Trino ................................................................................................. 137

Pulsar SQL 的工作原理 ...................................................................................... 138

配置 Pulsar SQL ................................................................................................. 140

性能考量 ............................................................................................................ 142

小结 ................................................................................................................... 143

第 11 章　部署 Pulsar ........................................................................ 144

Docker ................................................................................................................ 144

裸金属 ................................................................................................................ 146

最低要求 ..................................................................................................... 146

开始安装 ..................................................................................................... 146

部署 ZooKeeper........................................................................................... 147

启动 BookKeeper ........................................................................................ 148

启动 Pulsar .................................................................................................. 148

公有云厂商 ........................................................................................................ 149

AWS ............................................................................................................ 151

Azure ........................................................................................................... 152

Google Cloud Platform ................................................................................ 153

Kubernetes ......................................................................................................... 153

小结 ................................................................................................................... 155

第 12 章　运维 Pulsar ........................................................................ 156

Apache BookKeeper 指标 .................................................................................. 157

服务器指标 ................................................................................................. 157

Journal 指标 ................................................................................................ 158

存储指标 ..................................................................................................... 159

Apache ZooKeeper 指标 ..................................................................................... 159

服务器指标 ................................................................................................. 160

请求指标 ..................................................................................................... 161

主题指标 ............................................................................................................ 161

消费者指标 ........................................................................................................ 163

Pulsar 事务指标 ................................................................................................. 163

Pulsar Functions 指标 ......................................................................................... 164

高级运维技巧 .................................................................................................... 165

拦截器和追踪机制 ...................................................................................... 165

Pulsar SQL 指标 .......................................................................................... 167

指标转发 ............................................................................................................ 169

监控面板 ..................................................................................................... 169

小结 ................................................................................................................... 170

第 13 章　未来 ................................................................................... 171

编程语言支持 .................................................................................................... 171

扩展接口 ..................................................................................................... 171

增强 Pulsar Functions .................................................................................. 172

架构简化 / 扩展 ........................................................................................... 175

消息平台桥接器 .......................................................................................... 177

小结 ................................................................................................................... 180

附录 A Pulsar Admin API ................................................................. 181

附录 B Pulsar Admin CLI ................................................................. 189

附录 C　跨地域复制 ........................................................................... 192

附录 D Pulsar 中的安全、认证和鉴权 ............................................... 198

关于作者 ............................................................................................ 202

封面介绍 ............................................................................................ 202

查看详情

精通Apache Pulsar：可伸缩云原生事件流实践

内容简介:

作者简介:

目录: