Sentry 监控 - Snuba 数据中台本地开发环境配置实战-黑客24小时在线接单的网站

本文转载自微信公众号「黑客下午茶」，作者少。请联系黑客下午茶微信官方账号转载本文。

克隆仓库

分别克隆 getsentry/sentry 与 getsentry/snuba：

git clone https://github.com/getsentry/sentry.git

git clone https://github.com/getsentry/snuba.git

依赖(以 )安装系统Mac 为例)

Xcode CLI tools

xcode-select--install

Brewfile

进入 sentry 文件夹，你会看到 Brewfile 文件：

cdsentry

Brewfile

#requiredtorundevservices

cask'docker'

brew'pyenv'

#requiredforpyenv'spython-build

brew'openssl'

brew'readline'

#requiredforyarntest-u

brew'watchman'

#requiredtobuildsomeofsentry'sdependencies

brew'pkgconfig'

brew'libxslt'

brew'libxmlsec1'

brew'geoip'

#CurrentlyneededbecauseonBigSurthere'snowheelforit

brew'librdkafka'

#direnvisn'tdefinedhere,becausewehaveitconfiguredtocheckforabootstrappedenvironment.

#Ifit'sinstalledintheearlystepsofthesetupprocess,itjustleadstoconfusion.

#brew'direnv'

tap'homebrew/cask'

#requiredforacceptancetesting

cask'chromedriver'

如果你已经在当地安装了 Docker Desktop 并且已经启动，可以cask 'docker 注释掉。

接下来，操作：

brewbundle--verbose

如果你以前没有 Docker Desktop，还需要手动启动：

open-g-aDocker.app

构建工具链

Sentry 依赖于 Python Wheels(包括二进制扩展模块的包)，官方分发给以下平台:

Linux 兼容 PEP-513 (manylinux1)
macOS 10.15 或更高版本

如果您的开发机器没有运行上述系统之一，则需要安装 Rust 工具链https://www.rust-lang.org/tools/install 安装编译器及相关工具。安装后，Sentry 安装程序将自动使用 Rust 无需额外配置，构建所有二进制模块。

官员通常会跟踪最新的稳定性 Rust 版本，每六周更新一次。因此，请确保您的偶尔运行Rust 工具链保持最新：

rustupupdatestable

Python

Sentry 使用 pyenv 安装管理 Python 版本brew bundle 安装时。

需要安装所需版本Python，您需要操作以下命令。这将需要一段时间，因为您的计算机实际上正在编译 Python!

makesetup-pyenv

这里假设你是 Zsh 用户。

如果您输入 which python，你应该看到类似的 $HOME/.pyenv/shims/python 而不是 ///usr/bin/python 内容。这是因为您的启动脚本中添加了以下内容：

cat ~/.zprofile，您将看到以下内容：

#MacPortsInstalleradditionon2021-10-20_at_11:48:22:addinganappropriatePATHvariableforusewithMacPorts.

exportPATH="/opt/local/bin:/opt/local/sbin:$PATH"

#FinishedadaptingyourPATHenvironmentvariableforusewithMacPorts.

#ItisassumedthatpyenvisinstalledviaBrew,sothisisallweneedtodo.

eval"$(pyenvinit--path)"

虚拟环境

你现在准备好创建 Python 虚拟环境。

python-mvenv.venv

并激活虚拟环境：

source.venv/bin/activate

如果一切正常，操作 which python 现在应该导致 //Users/you/sentry/.venv/bin/python 的结果。

Snuba 配置实战

启动 Snuba 相关依赖项容器

cd../sentry

gitcheckoutmaster

gitpull

source.venv/bin/activate

sentrydevservicesup--exclude=snuba

#11:17:59[WARNING]sentry.utils.geo:settings.GEOIP_PATH_MMDBnotconfigured.

#11:18:01[INFO]sentry.plugins.github:apps-not-configured

#>Pullingimage'postgres:9.6-alpine'

#>Pullingimage'yandex/clickhouse-server:20.3.9.70'

#>Notstartingcontainer'sentry_relay'becauseitshouldbestartedon-demandwithdevserver.

#>Creating'sentry_redis'volume

#>Creating'sentry_zookeeper_6'volume

#>Creating'sentry_kafka_6'volume

#>Creatingcontainer'sentry_redis'

#>Creatingcontainer'sentry_zookeeper'

#>Creatingcontainer'sentry_kafka'

#>Startingcontainer'sentry_redis'(listening:('127.0.0.1',6379))

#>Startingcontainer'sentry_kafka'(listening:('127.0.0.1',9092))

#>Startingcontainer'sentry_zookeeper'

#>Creating'sentry_clickhouse'volume

#>Creatingcontainer'sentry_clickhouse'

#>Creating'sentry_postgres'volume

#>Creating'sentry_wal2json'volume

#>Startingcontainer'sentry_clickhouse'(listening:('127.0.0.1',9000),('127.0.0.1',9009),('127.0.0.1',8123))

#>Creatingcontainer'sentry_postgres'

#>Startingcontainer'sentry_postgres'(listening:('127.0.0.1',5432))

这将在 master 获得最新版本的 Sentry，并调出所有 snuba 依赖项。

Snuba 主要依赖 clickhouse，zookeeper，kafka，redis 相关容器。

docker ps 检查：

1149a6f6ff23postgres:9.6-alpine"docker-entrypoint.s…"3minutesagoUp3minutes127.0.0.1:5432->5432/tcpsentry_postgres

a7f3af7d52bbyandex/clickhouse-server:20.3.9.70"/entrypoint.sh"3minutesagoUp3minutes127.0.0.1:8123->8123/tcp,127.0.0.1:9000->9000/tcp,127.0.0.1:9009->9009/tcpsentry_clickhouse

68913ee15c43confluentinc/cp-zookeeper:6.2.0"/etc/confluent/dock…"3minutesagoUp3minutes2181/tcp,2888/tcp,3888/tcpsentry_zookeeper

5a248eb26ed3confluentinc/cp-kafka:6.2.0"/etc/confluent/dock…"3minutesagoUp3minutes127.0.0.1:9092->9092/tcpsentry_kafka

0573aff7b5afredis:5.0-alpine"docker-entrypoint.s…"3minutesagoUp3minutes127.0.0.1:6379->6379/tcpsentry_redis

设置 Snuba 虚拟环境

cdsnuba

makepyenv-setup

python-mvenv.venv

source.venv/bin/activate

pipinstall--upgradepip==21.1.3

makedevelop

查看迁移列表

snubamigrationslist

#system

#[]0001_migrations

#events

#[]0001_events_initial

#[]0002_events_onpremise_compatibility

#[]0003_errors

#[]0004_errors_onpremise_compatibility

#[]0005_events_tags_hash_map(blocking)

#[]0006_errors_tags_hash_map(blocking)

#[]0007_groupedmessages

#[]0008_groupassignees

#[]0009_errors_add_http_fields

#[]0010_groupedmessages_onpremise_compatibility(blocking)

#[]0011_rebuild_errors

#[]0012_errors_make_level_nullable

#[]0013_errors_add_hierarchical_hashes

#[]0014_backfill_errors(blocking)

#[]0015_truncate_events

#transactions

#[]0001_transactions

#[]0002_transactions_onpremise_fix_orderby_and_partitionby(blocking)

#[]0003_transactions_onpremise_fix_columns(blocking)

#[]0004_transactions_add_tags_hash_map(blocking)

#[]0005_transactions_add_measurements

#[]0006_transactions_add_http_fields

#[]0007_transactions_add_discover_cols

#[]0008_transactions_add_timestamp_index

#[]0009_transactions_fix_title_and_message

#[]0010_transactions_nullable_trace_id

#[]0011_transactions_add_span_op_breakdowns

#[]0012_transactions_add_spans

#discover

#[]0001_discover_merge_table

#[]0002_discover_add_deleted_tags_hash_map

#[]0003_discover_fix_user_column

#[]0004_discover_fix_title_and_message

#[]0005_discover_fix_transaction_name

#[]0006_discover_add_trace_id

#[]0007_discover_add_span_id

#outcomes

#[]0001_outcomes

#[]0002_outcomes_remove_size_and_bytes

#[]0003_outcomes_add_category_and_quantity

#[]0004_outcomes_matview_additions(blocking)

#metrics

#[]0001_metrics_buckets

#[]0002_metrics_sets

#[]0003_counters_to_buckets

#[]0004_metrics_counters

#[]0005_metrics_distributions_buckets

#[]0006_metrics_distributions

#[]0007_metrics_sets_granularity_10

#[]0008_metrics_counters_granularity_10

#[]0009_metrics_distributions_granularity_10

#[]0010_metrics_sets_granularity_1h

#[]0011_metrics_counters_granularity_1h

#[]0012_metrics_distributions_granularity_1h

#[]0013_metrics_sets_granularity_1d

#[]0014_metrics_counters_granularity_1d

#[]0015_metrics_distributions_granularity_1d

#sessions

#[]0001_sessions

#[]0002_sessions_aggregates

#[]0003_sessions_matview

运行迁移

snubamigrationsmigrate--force

#......

#2021-12-0119:45:57,557Runningmigration:0014_metrics_counters_granularity_1d

#2021-12-0119:45:57,575Finished:0014_metrics_counters_granularity_1d

#2021-12-0119:45:57,589Runningmigration:0015_metrics_distributions_granularity_1d

#2021-12-0119:45:57,610Finished:0015_metrics_distributions_granularity_1d

#2021-12-0119:45:57,623Runningmigration:0001_sessions

#2021-12-0119:45:57,656Finished:0001_sessions

#2021-12-0119:45:57,669Runningmigration:0002_sessions_aggregates

#2021-12-0119:45:57,770Finished:0002_sessions_aggregates

#2021-12-0119:45:57,792Runningmigration:0003_sessions_matview

#2021-12-0119:45:57,849Finished:0003_sessions_matview

#Finishedrunningmigrations

检查迁移

进入 Clickhouse 容器：

dockerexec-itsentry_clickhouseclickhouse-client

#运行如下`sql`语句：

selectcount()fromsentry_local

#ClickHouseclientversion20.3.9.70(officialbuild).

#Connectingtolocalhost:9000asuserdefault.

#ConnectedtoClickHouseserverversion20.3.9revision54433.

#a7f3af7d52bb:)selectcount()fromsentry_local

#SELECTcount()

#FROMsentry_local

#┌─count()─┐

#│0│

#└─────────┘

#1rowsinset.Elapsed:0.008sec.

#a7f3af7d52bb:)

查看相关实体数据集

snubaentitieslist

#DeclaredEntities:

#discover

#events

#groups

#groupassignee

#groupedmessage

#metrics_sets

#metrics_counters

#metrics_distributions

#outcomes

#outcomes_raw

#sessions

#org_sessions

#spans

#transactions

#discover_transactions

#discover_events

启动服务器开发

此命令将启动 api 和所有 Snuba 消费者以从 Kafka 摄取数据：

snubadevserver

转到 http://localhost:1218/events/snql，你会看到一个简单的查询 UI。

« 2024年9月 »
一	二	三	四	五	六	日
						1
2	3	4	5	6	7	8
9	10	11	12	13	14	15
16	17	18	19	20	21	22
23	24	25	26	27	28	29
30